Deep Compression Vector Quantize AutoEncoder?

[markson14]

It's a very impressive job! Well done.

I am wondering if you have conducted any further experiments on vector quantization. The DCAE-f128 can compress a 256x256 image into a 2x2 feature map, resulting in 4 tokens with VQ. This could lead to significant acceleration in LLM training and inference, paving the way for real-time video generation. Feel free to ask if you need any more adjustments!

[han-cai]

Thanks for your interest in our work! VQ is one direction we are working on. We will push our updates to this repo.

[markson14]

Thanks for your interest in our work! VQ is one direction we are working on. We will push our updates to this repo.

That's wonderful! We are trying to train the DCAE with our own dataset. I wonder how many epochs we should use and what learning rate is recommended, as these details were not mentioned in the 4.1 implementation details.