4.0 Env - Affine Lock - New Potential Benchmark

[Kinvert]

Affine Lock:

Player is given 2bytes randomized, and must use 8 actions (shuffle right, left, swap byes, swap nibbles, etc) to match the target 2bytes.

System starts with target, does shuffles (applies one action each time to the bytes) to a certain depth (up to 16 right now, like 16 shuffles) and makes sure it can't be solved in fewer steps than that depth.

Kinda similar to a Rubiks cube.

This could be a good benchmark. The difficulty can scale out to near impossible at depth 16. PufferLib 4.0 can get around depth 5 or 6.

More instructions in last comment in this PR.

[Kinvert]

ini profiles to probably be removed soon. These were for benchmarking, making sure runs do well with same hypers after changes etc. Probably won't be merged.

A lot of this is unit tests and tools. These can be removed before merge itself. The only real peculiarity off the top of my head is the 4MB bin which is precompiled shuffles of the bits the agents solve.

[Kinvert]

ok, this is pretty close. It's meant to be helpful for benchmarking. The tests can be removed right before merge.

This is based on "shuffling" and so it's possible to do 8 shuffles but have a game that is solveable in say 6. So a system was created to pre-compute games to a known minimum move win depth. This creates a bin file (5ish MB). So the bin file in here can be deleted, and people can generate it using the README instructions.

Human playable, though winning past depth 4 is very hard. PufferLib 4 is solving this to around depth 5. It can go to depth 16.

A user can relatively easily add a depth 7 game set to go between depth 6 and depth 8 for example, if 8 is a major cliff. Again this can go as written to depth 16, which is insanely insanely hard and not reachable with PufferLib 4.0.

If you want it more sparse you can get rid of rewards for lower depths.

[Kinvert]

Potential improvements:

• Log depth_D_attempts, it's already calculated and can potentially indicate what depths might be memorized
• Experimenting with step_grace (step_grace == 0 requires perfect optimal solve) scaffolding already exists
• Can potentially expand the horizontal size, like the number of bits in the affine lock (maybe a big change though)
• Add curriculum modes in the ini. I think this game can have multiple versions. Right now it has to solve sequentially or start over
• Put extra time in the generator and docs to make sure it's effective and easy to use
• Improve perf weighting, for example I experimented with Fibonacci weights today and it seems to get better results
• Build in test train isolation, keep in mind D16 seems to have only 100,548 perfect solutions mathematically
• Maybe add ini line where we choose which perf weights to use, Fibonacci, D^2, linear (current), etc

[Kinvert]

This was brought in to 5.0 here 2da3f7b and now that I know there is interest in the env I'm doing the cleanup from this vibe coded env to something clean enough to merge, which is currently here #645