# Another attempt at file compression :S

#### SoraK05

##### Well-Known Member
OP
Member
An idea came to me about a file having what I would call a 'magic number'.

Let's take these 5 bytes:
10 30 54 62 81

To calculate the magic number, you add the first byte, subtract the second, add the third, subtract the fourth, and so on.
It keeps adding/subtracting until it gets to the last byte, and the result is the magic number.

+10 - 30 +54 -62 +81
=53
53 is the magic number for those 5 bytes.

I believe no combination of those 5 bytes except that one will produce that magic number.
[EDIT was wrong about that.. ]

To take this idea further, perhaps the amount of each byte can be recorded:
i.e. 10 00s, 12 01s, 5 02s, 4 03s could look like this:

10 12 05 04

These 4 bytes represent the 31 bytes here.

The idea is to shuffle those known bytes, calculating each combination, until it reaches the magic number.

I may be wrong, but I believe the likelihood of having the magic number repeating in those combinations is low.

On top of that, a hash can be saved of the file, so when a magic number is matching, it verifies the contents with the hash.

I presume the likelihood of getting a matching magic number is generally low, and having a string that matches the magic number and hash is even lower.

It may take a really long time though :S

EDIT
Just realized that there are at least 11 other matching magic numbers to those 5 bytes :S
10 - 30 + 81 - 62 + 54 = 53
10 - 62 + 54 - 30 + 81 = 53
10 - 62 + 81 - 30 + 54 = 53
54 - 30 + 10 - 62 + 81 = 53
54 - 30 + 81 - 62 + 10 = 53
54 - 62 + 10 - 30 + 81 = 53
54 - 62 + 81 - 30 + 10 = 53
81 - 30 + 10 - 62 + 54 = 53
81 - 30 + 54 - 62 + 10 = 53
81 - 62 + 10 - 30 + 54 = 53
81 - 62 + 54 - 30 + 10 = 53

It may not be as few as I thought.. :S

Trying to confirm if the number of matching magic number and hashes at the same time is low enough to record as its own 'combination value' to make it worth it as a file smaller than the original one..

EDIT
A friend of mine gave a mathematical calculation to show that every 81 bytes + has more than one matching magic number and SHA1 at the same time, and the amount of matches grows exponentially the more bytes after that.
So, it seems the magic number + SHA1 hash combination (i.e. 20 bytes) may or may not have a collision for every 80 bytes.

1 person

#### FAST6191

##### Techromancer
Editorial Team
What you described is a minor tweak on bytesum aka second least advanced method of making a hash/checksum (the easiest being parity).

It is slightly better than your last idea of using a cryptographic grade hash (which by definition should not be able to be used for searching and data reconstruction purposes owing to the cascade effect beyond actually checking you got it right which as was mentioned in your previous thread is still not ideal) and brute force but you still seem to be heading towards/slowly reinventing the idea of data recovery parity (not to be confused with the basic concept of parity) as seen in the likes of the higher levels of RAID, PAR2 files and to a lesser extent in ECC ram.
Here each part of the file would be sampled more than necessary (oversampled) to recreate the file (you can define a straight line with two points but get four points and if one is wrong you still have the line) usually via something like a Fourier transform and then you can stick your processing power on recreating the file.

I am pretty much echoing the last thread though so I will stop there.

#### Blood Fetish

##### Quis custodiet ipsos custodes?
Member
I appreciate your enthusiasm on compression algorithms, but you cannot losslessly compress random data. The only way to achieve lossless compression is through pattern matching, which is in and of itself not random.

edit: There is a reason why hashing is often referred to as "one-way encryption." It is not possible to derive with certainty the original message upon which the hash was based.

#### SoraK05

##### Well-Known Member
OP
Member
I've been trying something.

Let's use a 1 byte hash and 1 byte magic number.
Let's say:

Hash = FF
Magic number = FF (+ or - 256, it doesn't matter whether it is + or -, as long as it is 256).

I've been trying to go through the 'sequence' of this combination by hand
So far I got
1. FF
2. FF 00
3. 01 00 FE

I believe that is the 'sequence' so far..

I kinda stopped there because it will take a while to figure out all the 3 byte matches.

I may be wrong, but this match seems to be seldom..

Unless I'm mistaken, '3' in the sequence == 01 00 FE

3 bytes so far
FF FF 03
That would be hash FF magic FF and sequence 03

I know this still represents 3 bytes, but it may take me a while to figure out how many matches there are for the 3 byte combinations and more that match the hash and sequence, to have the 'hash/magic match "sequence"'. So far it appears to be seldom.

So far the magic number has seldom matches in proportion to the hash - at least up to 2 bytes.

Perhaps when the sequence is FF it represents a file >= 4 bytes (at least I'm hoping), potentially breaking even.

I'll try figure out how many matches there are with 3 bytes.
I'm also taking into account that this works for files starting with 01 or more.

Replies
0
Views
1K
Replies
1
Views
930
Replies
19
Views
3K
Replies
98
Views
20K
Replies
0
Views
599

1. 20K views

## Majora’s Mask PC port 2Ship2Harkinian gets its first release

After several months of work, the Harbour Masters 64 team have released their first public build of 2Ship2Harkinian, a feature-rich Majora's Mask PC port. This comes...
2. 18K views

## Mario Builder 64 is the N64's answer to Super Mario Maker

With the vast success of Super Mario Maker and its Switch sequel Super Mario Maker 2, Nintendo fans have long been calling for "Maker" titles for other iconic genres...
3. 17K views

## Modders hint at potential kernel exploit hack for Xbox One consoles

It's been a while since Microsoft released the Xbox One, and despite its age, there haven't been any reliable softmod methods to hack the console. Until now. A post...
4. 16K views

## The founder of Oculus is releasing a \$199 FPGA Game Boy system

Palmer Luckey is known for his pursuits into the world of virtual reality, having founded Oculus and designed the Rift VR headset. Prior to the \$2 billion dollar...
5. 14K views

## Ubisoft reveals 'Assassin's Creed Shadows' which is set to launch later this year

Ubisoft has today officially revealed the next installment in the Assassin's Creed franchise: Assassin's Creed Shadows. This entry is set in late Sengoku-era Japan...
6. 14K views

## Nintendo takes down the Breath of the Wild randomizer mod from Gamebanana

Another day, another Nintendo DMCA takedown against fan-made content. Just a few minutes ago, Nintendo issued a DMCA takedown notice against a widely known and...
7. 14K views

## RetroArch is now available in the Apple Store for iOS devices

Another day, another great emulator that makes its way into the Apple Store for more users to enjoy. With Apple opening its store up to videogame emulators earlier...
8. 14K views

## The Kingdom Hearts games are coming to Steam

After a little more than three years of exclusivity with the Epic Games Store, Square Enix has decided to bring their beloved Kingdom Hearts franchise to Steam. The...
9. 11K views

## Select PlayStation 2 games are coming to PlayStation 5

Sony is once more attempting to reintroduce players to their older library of games by re-releasing classic PlayStation 2 titles onto the PlayStation Store. During...
10. 11K views

## Skyward Sword HD randomizer announced with release date and trailer

Skyward Sword is a divisive title in the Zelda series. Hailed with praise at launch with a 93 Metacritic average, the game since received criticism for the...
1. 169 replies

## The founder of Oculus is releasing a \$199 FPGA Game Boy system

Palmer Luckey is known for his pursuits into the world of virtual reality, having founded Oculus and designed the Rift VR headset. Prior to the \$2 billion dollar...
2. 108 replies

## Modders hint at potential kernel exploit hack for Xbox One consoles

It's been a while since Microsoft released the Xbox One, and despite its age, there haven't been any reliable softmod methods to hack the console. Until now. A post...
3. 108 replies

## Majora’s Mask PC port 2Ship2Harkinian gets its first release

After several months of work, the Harbour Masters 64 team have released their first public build of 2Ship2Harkinian, a feature-rich Majora's Mask PC port. This comes...
4. 104 replies

## Nintendo takes down the Breath of the Wild randomizer mod from Gamebanana

Another day, another Nintendo DMCA takedown against fan-made content. Just a few minutes ago, Nintendo issued a DMCA takedown notice against a widely known and...
5. 98 replies

## Ubisoft reveals 'Assassin's Creed Shadows' which is set to launch later this year

Ubisoft has today officially revealed the next installment in the Assassin's Creed franchise: Assassin's Creed Shadows. This entry is set in late Sengoku-era Japan...
6. 91 replies

## The Kingdom Hearts games are coming to Steam

After a little more than three years of exclusivity with the Epic Games Store, Square Enix has decided to bring their beloved Kingdom Hearts franchise to Steam. The...
7. 77 replies

## Select PlayStation 2 games are coming to PlayStation 5

Sony is once more attempting to reintroduce players to their older library of games by re-releasing classic PlayStation 2 titles onto the PlayStation Store. During...
8. 66 replies

## Mario Builder 64 is the N64's answer to Super Mario Maker

With the vast success of Super Mario Maker and its Switch sequel Super Mario Maker 2, Nintendo fans have long been calling for "Maker" titles for other iconic genres...
9. 64 replies

## PlayStation State of Play May 2024 showcase - God of War: Ragnarok coming to PC

The latest State of Play is here. This is PlayStation's Summer showcase, providing updates to new updates on upcoming games and brand new reveals. The 35-minute...
10. 62 replies

## Summer Game Fest 2024 coverage - Civilization VII announced

E3 may be gone, but it's not forgotten, as the trend of a massive Summer video game showcase still lives on in the form of the Summer Game Fest. Promising two hour of...

## Is grabbing an iPhone SE worth it?

### Recent Content

12. #### Help with kids android tablet

Help Users
• No one is chatting at the moment.
• @ Psionic Roshambo:
Float like a butterfly burns when I pee lol
+1
• @ BigOnYa:
So does that mean your date was not good? It burns now?
+1
• @ K3Nv2:
Got two new stds in one night
+1
• @ BigOnYa:
Giggity
+1
• @ The Catboy:
I don't bite! Minus the times when I did bite
• @ The Catboy:
Like 5 minutes ago
• @ K3Nv2:
Billie needs her lunch
• @ K3Nv2:
Ffs papa brought back the cheeseburger pizza it's like the only decent pie they had since the 80s
• @ BigOnYa:
I'm not a fan of papa johns, but that does sound good. We hardly order out pizza, I like making my own, but when we do its donatoes
• @ K3Nv2:
I get them like once every two months anymore
• @ K3Nv2:
Just because it's half a mile from where I live
• @ BigOnYa:
Request next time you order, that Shaq deliver it to you
• @ K3Nv2:
I want him to buy me a chain also
• @ K3Nv2:
Open it right next to the one we have
+1
• @ BakerMan:
guys should i make a new thread and just count the amount of posts until kyle, luke or leo joins the thread for fun?
• @ BakerMan:
kyle's fine, just waiting for that wario joke

luke and leo though, they yap until the thread's enjoyability is about halved
• @ K3Nv2:
Leo is Luke's alterego when he gets hard
• @ BigOnYa:
Luke is gone, he got banned. And I'm surprised Leo hasn't yet
• @ K3Nv2:
Subway was actually pretty decent tonight
• @ BigOnYa:
Wut you get, a seafood and psi salad sub
• @ K3Nv2:
Psi had my footlong meatball special
+1
• @ PandaPandel:
i want a meatball sandwich
now
• @ K3Nv2:
Gay
• @ BigOnYa:
Bout time you came out and admitted it
• @ K3Nv2:
Bigonya talks to himself often
+1
@ K3Nv2: Bigonya talks to himself often +1