Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(WiiU) Update WUT and add RetroArch target for WUHB files (Aroma enviroment) #14429

Open
ShadowOne333 opened this issue Sep 18, 2022 · 61 comments

Comments

@ShadowOne333
Copy link
Contributor

ShadowOne333 commented Sep 18, 2022

Description

A few weeks ago, the WiiU received a new form of CFW called "Aroma", which runs apps under the WUHB extension.
What this is effectively creating a coldboot option for the system, and also adds a bunch of features and other things as plugins directly on boot.

One of said options is the ability to be able to run Homebrew apps straight from the WiiU's home menu, as if it they were official apps on the system. This way, each homebrew app can be threated as their own app within the system.

In order to make RetroArch compatible with this setup, an update of WUT is required, as well as having a TV logo image (tv_splash.png, 1280x720), the gamepad image (drc_splash.png, 854x480) and the home menu image (icon.png, 132x179).
With that, creating the WUHB file out of the rpx should be easy enough by following these recommended instructions after updating both wut and wut-tools:
https://gbatemp.net/threads/release-aroma-environment-for-wii-u.618474/page-17#post-9940681

Expected behavior

Updating to the Aroma-friendly WUHB format would allow for RetroArch to be recognized and properly loaded on the system, alongside their proper boot logo, icon, and message.

Actual behavior

RetroArch currently cannot run under this environment since; while Aroma does have support for RPX apps, upon loading it simply stays on a black screen.

@vgmoose
Copy link
Contributor

vgmoose commented Sep 20, 2022

Maschell added the following info in the Aroma discord about rpx-loading:

Retroarch currently loads a core by loading into a special area in the memory and then using some old HBL magic to launch it. With aroma we can't use this region, because it's reserved for aroma and it's plugins. You can load a .rpx via Aroma though, Mocha_LaunchRPX is too "low" level, but librxpload (https://github.com/wiiu-env/librpxloader) could be extended to load a different .rpx which is inside the .wuhb.

@grockk
Copy link

grockk commented Sep 26, 2022

I wonder if the vWii retroarch could be updated to support the sign_c2w_patcher in a forwarder or inject. We'd be limited to 480p but for retro games on Wii U that seems like it'd work pretty good enough. And then it would be compatible with Aroma.

@ashquarky
Copy link
Contributor

I think I'm going to have a go at a wut port, at least depending on my availability over the holidays... I wrote up a bit of a game plan that might work but there are a few key questions:

  • Merging changes to all the cores and the frontend and the docker image at the same time is going to be a pain. Do we try and make a new platform= (wiiu-wut alongside wiiu?) to ease that or do we just wait for a release and then merge immediately after? I would prefer the latter but I don't know how RetroArch's release cycle works
  • Should wut be added in-tree? Since all the builds run on Docker now I don't see a reason to (especially considering that wut versions are somewhat tied to devkitPPC versions) but since libogc is in-tree it's worth asking

cc @LibretroAdmin @gblues

@LibretroAdmin
Copy link
Contributor

LibretroAdmin commented Dec 18, 2022

wiiu-wut - as in a new platform/target - seems like a good idea. Overall I'm fine with whatever @gblues thinks we should do.

Should wut be added in-tree? Since all the builds run on Docker now I don't see a reason to (especially considering that wut versions are somewhat tied to devkitPPC versions) but since libogc is in-tree it's worth asking

How big is it ?

@ghost
Copy link

ghost commented Dec 22, 2022

I would prefer that we avoid adding WUT in-tree. We should consume WUT artifacts, and we can control the version we get without resorting to completely duplicating the source tree. Adding it in-tree creates a bunch of toil. If we needed to add functionality to WUT, it should be contributed upstream. Which yes, may take a little longer, but it means that more projects than just RetroArch benefit.

I am in favor of having a separate build target. I propose that we just call it wut (which already implies wiiu). I also propose that once all cores have successful wut builds that the wiiu target builds be left deprecated. This is because the whole point of WUT is to provide a stable dev toolkit and I have no interest in maintaining 2 builds indefinitely.

@gblues
Copy link
Contributor

gblues commented Dec 22, 2022

whoops, I was signed into my work github account. ^ that's me. sorry!

@ashquarky
Copy link
Contributor

I will mention that my current draft of RetroArch (frontend) does not keep both codepaths - it assumes wut.

Also, the differences in the cores might not be that bad - snes9x at least was just some (backwards-compatible) compiler flag changes; thus for that core a new target isn't needed. I assume other cores will be more complicated though.

(specifically -mwup needs to be replaced - I swear I've PRed this to some cores before but I can't find it now..)

@ashquarky
Copy link
Contributor

For those following along:

Right now it mostly works under Tiramisu, aside from some long load times. Toolchain container is not well-tested, and cores still need to be updated.

@ashquarky
Copy link
Contributor

Just submitted the makefile changes for snes9x - with luck, all the cores will be like this. The core now builds fine on both old and new devkitPPC (with and without wut) at the cost of some extra meaningless warnings on the old version.

@ShadowOne333
Copy link
Contributor Author

I didn't see the last messages, this all sounds fantastic.
Did the recent commit that got merged for the makefile changes add .wuhb cores for RetroArch on WiiU?
Or is that still WIP?

@ashquarky
Copy link
Contributor

As an update for everyone:

  • Basically everything is working except for Salamander, which bootloops for some reason
  • I've had to disable libfat for the SD card since many Aroma modules mount the SD using the Cafe driver, and it's unsafe to have two drivers mounting the same filesystem at once. This makes startup really slow so I may re-enable it on Tira only. Playlists might be broken due to sd: becoming fs:/vol/external01.

I didn't see the last messages, this all sounds fantastic.

Did the recent commit that got merged for the makefile changes add .wuhb cores for RetroArch on WiiU?

Or is that still WIP?

wuhbs aren't in for this initial porting stage, though it may end up being a solution to the loading time issue.. we'll see! I'll probably try and land initial support first.

@ShadowOne333
Copy link
Contributor Author

As an update for everyone:

* Basically everything is working except for Salamander, which bootloops for some reason

* I've had to disable libfat for the SD card since many Aroma modules mount the SD using the Cafe driver, and it's unsafe to have two drivers mounting the same filesystem at once. This makes startup really slow so I may re-enable it on Tira only. Playlists might be broken due to sd: becoming fs:/vol/external01.

I didn't see the last messages, this all sounds fantastic.
Did the recent commit that got merged for the makefile changes add .wuhb cores for RetroArch on WiiU?
Or is that still WIP?

wuhbs aren't in for this initial porting stage, though it may end up being a solution to the loading time issue.. we'll see! I'll probably try and land initial support first.

That's awesome! Really looking forward to what you come up with for the RA rework for WiiU. Hope it all goes well with the last tidbits of nuances

@Clownacy
Copy link

Clownacy commented Jan 24, 2023

  • I've had to disable libfat for the SD card since many Aroma modules mount the SD using the Cafe driver, and it's unsafe to have two drivers mounting the same filesystem at once. This makes startup really slow so I may re-enable it on Tira only.

That sounds familiar: devkitPPC's newlib port had its disk IO buffering disabled for the longest time, and it has only been re-enabled in a recent commit. Without buffering, SD card accesses are extremely slow. No update to devkitPPC has been released yet with this fix, so in the meantime IO buffering has to be explicitly enabled with setvbuf(fp, NULL, _IOFBF, BUFSIZ);. I've done this before in a port of mine, and it does greatly improve the SD card access times (including making the homebrew start-up much faster).

I hope this helps.

@ashquarky
Copy link
Contributor

It's a good shout - setvbuf does help a lot, and we're already doing it in the wiiu port. The issues are instead mostly from the hundreds of small files existing in the assets folder, as well as the directory listings RA does. (the dirent functions get slower if there are larger files in the folder? not more, but larger?)

...wait, why are we setting a 128k vbuf and then a 4k one straight after? that's not in a3be192, did we get screwed on a merge conflict?

@ashquarky
Copy link
Contributor

Also hey, had no idea devkitPPC had enabled buffering upstream. Do we know what their buffer sizes are and what release rev that change is in? Very cool if they've fixed one of the biggest pain points for porting things (fgetc)

@Clownacy
Copy link

From what I can make of newlib's code, the default buffer size is 1024 bytes (BUFSIZ). There doesn't appear to be any release of devkitPPC that has this change yet: buffering was enabled in August of last year, but the most recent release of devkitPPC is from May.

@ashquarky
Copy link
Contributor

Makes sense. Looks like we'll still have to setvbuf in RetroArch (64k and 128k are the ideal sizes for large files in Cafe) but it'll be a nice help for other projects.

@Ploggy
Copy link

Ploggy commented Mar 6, 2023

Hey all.
I've been chipping away at editing the Retroarch Cores Makefiles to add the necessary changes Quarky mentioned for Aroma support.
I have changed all the Cores that are listed here:
#14925

I manged to compile a handful of modified Cores and have tested them on WiiU to see how they perform.
There are a few Cores that fail to compile with the new changes but I guess that's to be expected and will have to be fixed on a per-core basis?
Good news is they do boot direct from the WiiU Menu and can load a rom :)
Bad news is Core switching seems to be broken and loading a rom from usb will throw an error.

Another problem (as mentioned above) is the long core load/rom load times, it can take up to 2 1/2 mins from loading the Core from WiiU Menu to game boot.
https://www.youtube.com/watch?v=jn8YIVMi4EA&feature=youtu.be

But even with current issues is great to see Retroarch Cores working in Aroma! the future is now! :D
Thank You Quarky!!!

@Sowden
Copy link

Sowden commented Mar 22, 2023

Hey @Ploggy, is there anywhere we could download this experimental wuhb to try it out? 😁

@Ploggy
Copy link

Ploggy commented Mar 22, 2023

https://www.mediafire.com/folder/bvrennhlwsw2s/Aroma+Cores
It isn't in Wuhb format just single rpx files but sure :)
I did create a Wuhb file for testing but Retroarch hasn't got the ability to load cores from within a packaged Whub.

EDIT: fixed link

@Sowden
Copy link

Sowden commented Mar 22, 2023

Thanks for the speedy response! Editing my first post cause I'm dumb, lol. I'll work with the Atari cores to see how well it will work, but it is nice to see it running in Aroma. Good job!

@Ploggy
Copy link

Ploggy commented Mar 22, 2023

The Atari Cores should be fine, just be warned the Cores do take a while to boot currently and so does rom loading ;)

@ashquarky
Copy link
Contributor

Cores do take a while to boot currently and so does rom loading ;)

I am pretty sure this is an Aroma-specific issue, by the way. I don't have a great fix for it right now though. If you're switching between Aroma and Tira, try deleting retroarch.cfg and see if that speeds it up - it'll remember that libfat was disabled (from Aroma) otherwise ;)

@Ploggy
Copy link

Ploggy commented Mar 23, 2023

(Testing Amstrad CPC Core)
On Tira CFW = Core swapping possible..
loading a Aroma Core in Retroarch takes 41 seconds
Once that Core is loaded, loading a game takes a further 41 seconds

On Aroma CFW (with fresh Retroarch.cfg) = Core swapping not possible.. When you try to load another Core after 54 seconds it'll return back to the Retroarch UI but the last Core will still be loaded.
loading Core from WiiU Menu takes 1 min 18 seconds.
Once that Core is loaded, loading a game takes a further 55 seconds.

So its a problem for both CFW but on Aroma Core switching is broken as well.

@ashquarky
Copy link
Contributor

Core switching was definitely working in my tests :(
Guess I'll have to check again...

@Ploggy
Copy link

Ploggy commented Mar 23, 2023

Closing content and browsing long romlists with thumbnails is alot slower too :( Its a similar situation we had in Retroarch a while back with slow io?

@Sowden
Copy link

Sowden commented Apr 10, 2023

https://easyupload.io/m4je0i It isn't in Wuhb format just single rpx files but sure :) I did create a Wuhb file for testing but Retroarch hasn't got the ability to load cores from within a packaged Whub.

Hey guys. So I did some testing on the link you shared and didn't get great results :\ First off, I wasn't able to get anything to boot in Aroma, other than the Retroarch main menu. I could move around the Retroarch menus just fine though. When I tried to load content or even exit the app to return to the Wii U main menu, it would crash. Now I read that it has some long loading times, so each test I gave it about 10 minutes, but each one froze on the screen. And secondly, it also locks when I'm trying to load a core. And then oddly it showed the information for the last rpx core I loaded. So say for example that I tried loading Hatari rpx first. It crashed on me so I restarted the Wii U. I then loaded the DOSBox rpx and I got to the Retroarch menu. If I look at the "Core Information" at the top it would display "Atari - ST (Hatari)" and display all of the information on the ST core. Even though at the bottom corner it shows that "DOSBox - SVN" is loaded. Very odd. However, loading the rpx files you gave me through Tira works perfectly fine. Does this sound right or am I possibly doing something wrong? And in a side note, if the problems are Aroma related, might it be time to get Marshall in on this conversation?

@Sowden
Copy link

Sowden commented Jun 8, 2023

Man, has progress on this stalled?

@Ploggy
Copy link

Ploggy commented Sep 7, 2023

Updated the link with the Test Aroma Cores since it was dead :) ^^^^

@harrysof
Copy link

harrysof commented Nov 4, 2023

still nothing on this ? :(

@CrossScarDev
Copy link

Any more work?

@Nahieluniversal
Copy link

Any news on the port?

@Ploggy
Copy link

Ploggy commented Mar 2, 2024

Kinda, I still don't know what the cause of my slow loading is if it doesn't happen in Quarky's tests.. tho it has been said that the issue with the slow loading could be remedied with a PR that apparently has a 100x performance increase in WiiU native FAT32 driver :D not a bad addition lol
devkitPro/wut#340
It's hoped that that will fix any slow loading but it hasn't been implemented yet. I'm hoping it can be added to both Tira Retroarch and Aroma Retroarch. :D

But still strange Quarky doesn't get the slow loading issue. But others here have :/ Perhaps I'm compiling it wrong? Quarky do you still have an Aroma Core lying around that you used for tests?

@SMwaterSHLDhelp
Copy link

Has there been any easy to use implementation yet?

@Ploggy
Copy link

Ploggy commented Jun 1, 2024

https://www.youtube.com/watch?v=lI2cNgmiosg&ab_channel=Ploggy

Aroma RA now Loads Cores/Games at a much more acceptable speed, I don't know what has changed between now and last year when I last tried it but I assume it must be Aroma CFW updates.?

@Clownacy
Copy link

Clownacy commented Jun 1, 2024

Beta 17's changelog mentions patching the OS to make readdir and stat faster. Maybe that is what has fixed the speed problem?

@Ploggy
Copy link

Ploggy commented Jun 1, 2024

Beta 17's changelog mentions patching the OS to make readdir and stat faster. Maybe that is what has fixed the speed problem?

I thought it could be that too :) It made a massive difference over what it was before.
It's what piqued my interest to try it again.

@Sowden
Copy link

Sowden commented Jun 1, 2024

Aroma RA now Loads Cores/Games at a much more acceptable speed, I don't know what has changed between now and last year when I last tried it but I assume it must be Aroma CFW updates.?

This looks great. Are the downloads for this still the link you posted above months ago?

@Ploggy
Copy link

Ploggy commented Jun 1, 2024

Aroma RA now Loads Cores/Games at a much more acceptable speed, I don't know what has changed between now and last year when I last tried it but I assume it must be Aroma CFW updates.?

This looks great. Are the downloads for this still the link you posted above months ago?

Yeah technically the Cores are the same but best to use this link.. It has a wuhb retroach file for a cleaner setup 👌

.. And a working Snes9x Core lol
https://www.mediafire.com/file/jgnw7qol59h00gp/Aroma_Cores_Extract_to_SD_Card.zip/file

Dont Try to load any Tiramisu Cores with this lol

EDIT: Fixed link :)

@vonmillhausen
Copy link

.. And a working Snes9x Core lol https://www.mediafire.com/file/ttevxgvdxddv76h/Aroma_Cores_Extract_to_SD_Card.zip/file

Sorry to bother you @Ploggy, but I get an error from MediaFire with that link indicating that maybe the file has been removed - did you take it down?

@Ploggy
Copy link

Ploggy commented Jun 3, 2024

.. And a working Snes9x Core lol https://www.mediafire.com/file/jgnw7qol59h00gp/Aroma_Cores_Extract_to_SD_Card.zip/file

Sorry to bother you @Ploggy, but I get an error from MediaFire with that link indicating that maybe the file has been removed - did you take it down?

Sorry here's a new link :)

https://www.mediafire.com/file/gcevyd36qgqtc5k/Aroma_Cores_Extract_to_SD_Card.zip/file

@vonmillhausen
Copy link

Sorry here's a new link :)

https://www.mediafire.com/file/gcevyd36qgqtc5k/Aroma_Cores_Extract_to_SD_Card.zip/file

You're a legend, thank you kindly! ❤️

@Sowden
Copy link

Sowden commented Jun 4, 2024

Hmmmmm. It appears that the Atari 5200 (A800) core crashes on loading a Atari atr floppy with a white text error screen. It looks like this is better, but not yet ready for primetime. And @Ploggy, could you please render and include a hitari core please? Those are the only two cores I use retroarch for anyways. Thanks man.

@Ploggy
Copy link

Ploggy commented Jun 4, 2024

Hmmmmm. It appears that the Atari 5200 (A800) core crashes on loading a Atari atr floppy with a white text error screen. It looks like this is better, but not yet ready for primetime. And @Ploggy, could you please render and include a hitari core please? Those are the only two cores I use retroarch for anyways. Thanks man.

Yea, its true this is all WIP right now and Core weirdness is to be expected. I'll try and recompile that Core and the Hatari one and post it here if there's any difference :)

@Ploggy
Copy link

Ploggy commented Jun 4, 2024

@Sowden I compiled Hatari and recompiled Atari800 but both seem the crash the same way? I dont know why?
Probably these Cores are particularly sensitive to the Makefile changes that Aroma need. I dont think this is something I can fix.. I'll have to leave these two Cores to the people working on porting Aroma :/ Sorry.

@Sowden
Copy link

Sowden commented Jun 4, 2024

@Sowden I compiled Hatari and recompiled Atari800 but both seem the crash the same way? I dont know why? Probably these Cores are particularly sensitive to the Makefile changes that Aroma need. I dont think this is something I can fix.. I'll have to leave these two Cores to the people working on porting Aroma :/ Sorry.

Hey man, don't shake it. Thanks for trying. I think from between the two different angles, the guys working on these cores, and the guys working on Aroma, their gonna meet these cores half way.

@PseudoDistant
Copy link

(Sorry if this seems a selfish ask)
Is there a WIP DOSbox core?

@ReeRee457
Copy link

I love the work you've been able to do, it's amazing.
Now, are there any plans to port over N64 and Sega Saturn cores? If so, then the Wii U could become an excellent emulation powerhouse.

@ReeRee457
Copy link

Sorry for sending another message so soon, but I have noticed a glitch with the SNES9X core. Some games, such as Super Metroid, does not boot. However, it does boot in the SNES9X 2010 core. I felt like bringing this issue to light.

@Bixthefin
Copy link

Bixthefin commented Jul 19, 2024

Aroma RA now Loads Cores/Games at a much more acceptable speed, I don't know what has changed between now and last year when I last tried it but I assume it must be Aroma CFW updates.?

This looks great. Are the downloads for this still the link you posted above months ago?

Yeah technically the Cores are the same but best to use this link.. It has a wuhb retroach file for a cleaner setup 👌

.. And a working Snes9x Core lol https://www.mediafire.com/file/jgnw7qol59h00gp/Aroma_Cores_Extract_to_SD_Card.zip/file

Dont Try to load any Tiramisu Cores with this lol

EDIT: Fixed link :)

I dont know much abouts this stuff but i recently tried it on my WiiU and it seems to be pretty stable and complete just asking if this is still being worked on and if this will become the main build for the WiiU soon. And or its own github page. aplogies if i said something that is not possible I am quite the newbie at this stuff.

@Ploggy
Copy link

Ploggy commented Jul 20, 2024

The plan it to eventually have the Aroma build be the main WiiU Retroarch port and is currently in progress :)

@deadlyjunk
Copy link

should we expect the aroma version to come out by the end of the year?

@Horizon531
Copy link

Aroma RA now Loads Cores/Games at a much more acceptable speed, I don't know what has changed between now and last year when I last tried it but I assume it must be Aroma CFW updates.?

This looks great. Are the downloads for this still the link you posted above months ago?

Yeah technically the Cores are the same but best to use this link.. It has a wuhb retroach file for a cleaner setup 👌

.. And a working Snes9x Core lol https://www.mediafire.com/file/jgnw7qol59h00gp/Aroma_Cores_Extract_to_SD_Card.zip/file

Dont Try to load any Tiramisu Cores with this lol

EDIT: Fixed link :)

Hi, sadly it seems the link is down again. Could you reupload it? Thank you for the work you're doing.

@Ploggy
Copy link

Ploggy commented Aug 15, 2024

Aroma RA now Loads Cores/Games at a much more acceptable speed, I don't know what has changed between now and last year when I last tried it but I assume it must be Aroma CFW updates.?

This looks great. Are the downloads for this still the link you posted above months ago?

Yeah technically the Cores are the same but best to use this link.. It has a wuhb retroach file for a cleaner setup 👌
.. And a working Snes9x Core lol https://www.mediafire.com/file/jgnw7qol59h00gp/Aroma_Cores_Extract_to_SD_Card.zip/file
Dont Try to load any Tiramisu Cores with this lol
EDIT: Fixed link :)

Hi, sadly it seems the link is down again. Could you reupload it? Thank you for the work you're doing.

Grab the link from here ;)

https://gbatemp.net/threads/retroarch-wiiu-wip.447670/post-10432540

@Davis8483
Copy link

Any update on when we'll see this as apart of an official release?

@LazaroFilm
Copy link

Bumping this back up because I would love this to be merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests