Cropping issue discussion

[nitinmukesh]

Arvrairobo commented 3 days ago

@nitinmukesh thank you for your PR, but it is not cropping the image, i have applied your PR and it is running successfully but it is not cropping the image and due to that, if there is a full image (not just face) then anitalker is not able to generate talking face. could you please recheck cropping code? so it should automatically crop the image and use that cropped image as an input correct? but it is not working like that

for following issue
ERROR: No matching distribution found for pytorch-lightning==1.6.5

python -m pip install pip==24.0

Make sure to install following
https://github.com/X-LANCE/AniTalker/blob/main/md_docs/run_on_windows.md

[nitinmukesh]

Test result with auto-crop

Images used

Output

1.mp4

2.mp4

3.mp4

5.mp4

[Arvrairobo]

@nitinmukesh thank you for the help, yes i figured out why it was not working so i missed two files

one was that .dat file and that .npy file, once i placed that in data_preprocess folder, it started to work thank you very much for your help

[Arvrairobo]

@nitinmukesh @newgenai79
can we achieve cropping with 512 X 512 resolution instead of 256X 256 ?

also i can see the blink and eyes movement are not very natural compares to echomimic, how we can achieve the same? i tried huber_pose as well but not getting perfect result.

what is hubert_full_control?

[nitinmukesh]

@nitinmukesh @newgenai79 can we achieve cropping with 512 X 512 resolution instead of 256X 256 ?

I don't think it's possible since the model is trained on 256 x 256 resolution.

also i can see the blink and eyes movement are not very natural compares to echomimic, how we can achieve the same?

EchoMimic is good as it is trained on large dataset, hence the results are better.

[Arvrairobo]

@nitinmukesh echomimic is very slow, for 40 secs video it took about 2.5 hours so not a practical solution. anitalker is very fast, thats why i am building a solution on top of it.

if author @liutaocode or any author can shed some lights on it or give some headstart on how to achieve blink and facial expression, that would be great, or if they release a code or more trained model that would also work, looking forward to it

[Arvrairobo]

still waiting for blink feature, also any update on the project? any new release of the code? or any new features?