Results 1 to 4 of 4

Thread: Finding duplicate or repetitive data in a binary file

  1. #1
    Senior Member

    Join Date
    Jan 2008
    Location
    A rock in the ocean
    Posts
    752
    Thanks
    99
    Thanked 135 Times in 79 Posts
    Rep Power
    290
    Reputation
    3356

    Default Finding duplicate or repetitive data in a binary file

    Hello all,
    Looking for a small utility to find repetitive or duplicate data (of possible lengths 128, 256, 512 or 1024 bytes) in a large binary file... any leads on an application that'll do that?



Look Here ->
  • #2
    Crazy Diamond
    Tiny's Avatar
    Join Date
    Dec 2010
    Location
    Tasmania
    Age
    64
    Posts
    6,391
    Thanks
    10,996
    Thanked 5,436 Times in 2,651 Posts
    Rep Power
    2156
    Reputation
    89057

    Default

    HxD will do that >>
    Cheers, Tiny
    "You can lead a person to knowledge, but you can't make them think? If you're not part of the solution, you're part of the problem.
    The information is out there; you just have to let it in."

  • #3
    Senior Member

    Join Date
    Jan 2008
    Location
    A rock in the ocean
    Posts
    752
    Thanks
    99
    Thanked 135 Times in 79 Posts
    Rep Power
    290
    Reputation
    3356

    Default

    Thanks Tiny - how? I knew HxD can search for a pattern that you know, but I want to find patterns that are unknown - for the software to find repetition of data itself. Can HxD still do that?

  • #4
    Crazy Diamond
    Tiny's Avatar
    Join Date
    Dec 2010
    Location
    Tasmania
    Age
    64
    Posts
    6,391
    Thanks
    10,996
    Thanked 5,436 Times in 2,651 Posts
    Rep Power
    2156
    Reputation
    89057

    Default

    No I don't think so. I would expect that it can't find unknown patterns, you need to give it a pattern to search.

    Forensic file searching is what you need to study.
    Finding repetitive unknown patterns in data files would require some sort of algorithm.

    These guys might know?
    Cheers, Tiny
    "You can lead a person to knowledge, but you can't make them think? If you're not part of the solution, you're part of the problem.
    The information is out there; you just have to let it in."

  • The Following User Says Thank You to Tiny For This Useful Post:

    mitaux8030 (04-05-18)

  • Bookmarks

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •