SuperNova 2 Support #86

DiscoStarslayer · 2024-02-06T21:02:40Z

I'm not the best pythoner in the world so some of this code might be cringe, however it is working properly for the following:

Features:

e-Amuse card registration and login
Last song selection storage
Options storage
Score storage
Calorie storage
Events

I still haven't done the following work:

Todo:

Expose functionality to frontend UI config (calorie display, events)
Identify why calorie values are not loading
Identify why last song not storing correctly for sn2 category
Doubles mode and course mode logic
Traffic gen implementation
Unit Tests

Shoutout to @987123879113 for sharing their research with me on the account binary format and the event logic, was a big help in making this work faster.

I plan on getting this a bit more cleaned up and upstream-able eventually, tackling the todos over time, however the branch is fully functional so it would be a shame to keep it to myself till then.

Open to any and all suggestions on the lack of quality on my python code, there was some fishy stuff with the inheritance that I'm not a big fan of but unsure what fits best with the project's vision.

Features: - e-Amuse card registration and login - Last song selection storage - Options storage - Score storage - Calorie storage - Events ToDo: - Expose functionality to frontend UI config (calorie display, events) - Identify why calorie values are not loading - Identify why last song not storing correctly for sn2 category - Doubles mode and course mode logic

DiscoStarslayer · 2024-02-06T21:03:48Z

bemani/backend/ddr/base.py

@@ -127,6 +127,13 @@ def format_profile(self, userid: UserID, profile: Profile) -> Node:
        """
        return Node.void("game")

+    def format_profile_part(self, userid: UserID, profile: Profile, part: str) -> Node:


Profile for SN2 is split into 3 binary chunks, where to break this logic down, it being here feels wrong

Honestly the format_profile and unformat_profile convention were both established to just give some sort of unity across games. They don't need to exist outside of the fact that the base of most games assumes it can call that for profile loads. If there's 3 parts here, this can diverge for cleaner code.

DiscoStarslayer · 2024-02-06T21:05:20Z

bemani/backend/ddr/ddrsn2/ddrsn2.py

+
+        return game
+
+    def handle_game_hiscore_request(self, request: Node) -> Node:


A lot of this stuff is most likely unused, I used the ddrx code as a baseline, need to take a look

Yeah, I would check whether these are even in the game at all, I would doubt it.

DiscoStarslayer · 2024-02-06T21:06:58Z

bemani/backend/ddr/ddrsn2/ddrsn2.py

+
+        return root
+
+    def handle_info_message_request(self, request: Node) -> Node:


Unsure where this API is used, figure it might get displayed to the user somehow on login but I couldn't work it out

This could be a string-encoded server message for setting flags? Dunno, that's how bishi does things somewhat.

Yeah the game logic seems to store the returned value to memory, but I haven't dug much further than that, I'll take a peek at the Bishi code, that might give me a few clues.

DiscoStarslayer · 2024-02-06T21:07:38Z

bemani/backend/ddr/ddrsn2/ddrsn2.py

+
+        return root
+
+    def handle_info_tenpo_request(self, request: Node) -> Node:


Looks to be some kind of remote-update functionality

DiscoStarslayer · 2024-02-06T21:08:45Z

bemani/backend/ddr/ddrsn2/playerinfo.py

+        idx = 0
+
+        # Empty option set is a non-zero opt array, unsure how to do this check with validated_dict cleanly?
+        if not profile.has_key("opt"):


Had to add new property to validated_dict to support this, maybe a cleaner way?

A ValidatedDict is just a subclass of a python dict, so you can just do if "opt" not in profile or similar. All of the functions on it are for access or insertion of storage where you want to constrain the type. Under the hood, it's just a Dict[str, object]

Thank you! Yes that is much cleaner and much more pythonic, I was looking for a class function but wasn't aware of the in syntax. 🙇

DiscoStarslayer · 2024-02-06T21:09:48Z

bemani/backend/ddr/ddrx.py

+from bemani.common import VersionConstants
+
+
+class DDRX(DDRBase):


I had to extract this to fix a circular dependency? Maybe not needed?

IDGI, where's the circular dependency? Does SN2 reference X for some reason? This isn't a problem anywhere else.

I might try my hand at it again because I think it might have had something to do with how I was importing modules instead of a file directly? But it was mainly a problem with the previous_version function.

How the code used to be:
In Stub: DDRX, DDRSN1
Own Module: DDRSN2

DDRX has a previous_version that points to DDRSN2 from the module, but DDRSN2 has a previous_version that points to DDRSN1 from the stub file. So the stub file needs the DDRSN2 import to work, but that import can't work until the stub file resolves for DDRSN1? That's how I understood it but my imports might have been strange too, so let me review this one a bit more.

Oh, it's because SN1 and X were in the stub file, so circular dependency. You moved the DDRX bits out of the stub into their own file, so it should work now.

DiscoStarslayer · 2024-02-06T21:14:19Z

bemani/data/mysql/music.py

@@ -105,8 +114,37 @@ def __get_musicid(self, game: GameConstants, version: int, songid: int, songchar
            },
        )
        if cursor.rowcount != 1:
-            # music doesn't exist
-            raise Exception(f"Song {songid} chart {songchart} doesn't exist for game {game} version {version}")
+            if game == GameConstants.DDR and version == VersionConstants.DDR_SUPERNOVA_2:


Not sure of a clean way to extract the song list from the game yet, so making them on demand here. There is probably a way to do it but the files on the hdd are encrypted so need to think of a method of extracting them.

You'll need to figure that out, or provide a song DB .tsv or something, because on-demand music stuff isn't supposed to be here. That's why this module has no way of getting the next ID. From it's perspective, the music DB is read-only.

I assume this also means no front-end? I think you mentioned that.

Yeah currently no front-end, but it's something I want to add so that the events can be managed among other things.

I wasn't aware of the .tsv files, this might be the approach I'll have to take here, there isn't any public tools for decrypting the dnas blobs that I know of and the easiest way is most likely to extract the data from memory once loaded.

DragonMinded · 2024-02-11T23:02:41Z

Hey, thanks for sending this in, this looks promising! Sorry about the slow response, I'm very burned out. I won't be able to merge this without at minimum traffic tests, and unit tests would be a nice bonus (but most non-core stuff doesn't have any since unit testing game responses tends to end up being a change inhibitor). Let me comment inline with your comments.

DragonMinded · 2024-02-11T23:04:49Z

bemani/backend/ddr/base.py

@@ -142,6 +149,24 @@ def unformat_profile(self, userid: UserID, request: Node, oldprofile: Profile) -
        """
        return oldprofile

+    def get_profile_by_refid_and_part(self, refid: Optional[str], part: Optional[str]) -> Optional[Node]:


Yeah, this definitely feels weird. But, the pattern is established. Worth a think.

DragonMinded · 2024-02-11T23:05:21Z

bemani/backend/ddr/ddrsn2/__init__.py

@@ -0,0 +1,3 @@
+from bemani.backend.ddr.ddrsn2.ddrsn2 import DDRSuperNova2


Interesting separating this into a module instead of a single file. Any reason for having this across a bunch of files? Don't see a problem with it, just curious.

Honestly, mostly just a leftover from my JS work. A common lint is only one class defined per file. If it's a bit awkward I'm ok with merging them into one file.

IDGAF either way, pattern in the repo is one file, we don't do silly things like have an import system that depends on there being one export/class per file, so you can get away with a lot of stuff.

DiscoStarslayer · 2024-02-12T01:06:51Z

Hey, thanks for sending this in, this looks promising! Sorry about the slow response, I'm very burned out.

Hey! Thank you very much for taking the time to review! I honestly didn't think I would get a response so quickly, so thanks a lot for that.

And understood with the requirements for merge, I agree with that and will work towards polishing this branch up. Game is already 17 years old so there is no urgency to any of this 😄

Going over your review now 👀

AlejandroGuzmanCL · 2024-11-08T20:21:44Z

Hello, sry for the bump but I was wondering if we can test this somehow, I havent found any docs or guide how to setup this locally.

DiscoStarslayer commented Feb 6, 2024

View reviewed changes

DragonMinded reviewed Feb 11, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SuperNova 2 Support #86

SuperNova 2 Support #86

DiscoStarslayer commented Feb 6, 2024

DiscoStarslayer Feb 6, 2024

DragonMinded Feb 11, 2024

DiscoStarslayer Feb 6, 2024

DragonMinded Feb 11, 2024

DiscoStarslayer Feb 6, 2024

DragonMinded Feb 11, 2024

DiscoStarslayer Feb 12, 2024

DiscoStarslayer Feb 6, 2024

DiscoStarslayer Feb 6, 2024

DragonMinded Feb 11, 2024

DiscoStarslayer Feb 12, 2024

DiscoStarslayer Feb 6, 2024

DragonMinded Feb 11, 2024

DiscoStarslayer Feb 12, 2024

DragonMinded Feb 12, 2024

DiscoStarslayer Feb 6, 2024

DragonMinded Feb 11, 2024

DiscoStarslayer Feb 12, 2024

DragonMinded commented Feb 11, 2024

DragonMinded Feb 11, 2024

DragonMinded Feb 11, 2024 •

edited

Loading

DiscoStarslayer Feb 12, 2024

DragonMinded Feb 12, 2024

DiscoStarslayer commented Feb 12, 2024

AlejandroGuzmanCL commented Nov 8, 2024


		return game

		def handle_game_hiscore_request(self, request: Node) -> Node:


		return root

		def handle_info_message_request(self, request: Node) -> Node:


		return root

		def handle_info_tenpo_request(self, request: Node) -> Node:

		from bemani.common import VersionConstants


		class DDRX(DDRBase):

		@@ -0,0 +1,3 @@
		from bemani.backend.ddr.ddrsn2.ddrsn2 import DDRSuperNova2

SuperNova 2 Support #86

Are you sure you want to change the base?

SuperNova 2 Support #86

Conversation

DiscoStarslayer commented Feb 6, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DragonMinded commented Feb 11, 2024

Choose a reason for hiding this comment

DragonMinded Feb 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DiscoStarslayer commented Feb 12, 2024

AlejandroGuzmanCL commented Nov 8, 2024

DragonMinded Feb 11, 2024 •

edited

Loading