Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PS3 Build #91

Open
humbertodias opened this issue Feb 15, 2024 · 39 comments
Open

PS3 Build #91

humbertodias opened this issue Feb 15, 2024 · 39 comments

Comments

@humbertodias
Copy link
Collaborator

I've notice that ps3 build were made through SCons and python.
https://github.com/kazzmir/paintown/releases/download/v3.6.0/paintown-ps3-3.6.0.pkg
Should we keep that way or try something new?

if so, do you remember steps?

I was looking at ps3toolchain : https://github.com/ps3dev/ps3toolchain

@humbertodias humbertodias closed this as not planned Won't fix, can't repro, duplicate, stale Feb 15, 2024
@juvinious
Copy link
Collaborator

You would need to build from the 3.6 source.
The instructions can be found under doc/ps3.
If it works/builds still is a wild guess though.

Allot of the mugen stuff was very preliminary and experimental in that release.

@humbertodias humbertodias reopened this Feb 15, 2024
@kazzmir
Copy link
Owner

kazzmir commented Feb 15, 2024

We should use meson to build. I doubt the ps3 stuff will work anymore. Maybe we could test it in an emulator or something if there are any good ones.

@humbertodias
Copy link
Collaborator Author

We should use meson to build. I doubt the ps3 stuff will work anymore. Maybe we could test it in an emulator or something if there are any good ones.

I have tested the release 3.6.0 on RPCS3 and it works well.
Screenshot 2024-02-15 at 10 37 00 PM

@humbertodias
Copy link
Collaborator Author

You would need to build from the 3.6 source. The instructions can be found under doc/ps3. If it works/builds still is a wild guess though.

Allot of the mugen stuff was very preliminary and experimental in that release.

Makes sense. I'm gonna try to build the pkg using docker as a cross compiler such as you did previously with mingw

@kazzmir
Copy link
Owner

kazzmir commented Feb 16, 2024

Thats cool that it works on RPCS3. The latest version of paintown now requires c++11 and threading support in gcc, so I'm not sure if the PS3 sdk PSL1ght supports that.

@humbertodias
Copy link
Collaborator Author

humbertodias commented Feb 17, 2024

Thats cool that it works on RPCS3. The latest version of paintown now requires c++11 and threading support in gcc, so I'm not sure if the PS3 sdk PSL1ght supports that.

It's almost working https://github.com/kazzmir/paintown/actions/runs/7941175624/job/21683244586#step:3:1

I have created the container with embedded ps3toolchain and it comes with compiled sdl1.2. Now it's needed to add the sdl2 and other paintown dependencies. Teorically, after that we should to be ready to compile and package.

@juvinious
Copy link
Collaborator

Once it's building and you are able to generate a pkg we should migrate the environment over from Scons to Meson.

@humbertodias
Copy link
Collaborator Author

Once it's building and you are able to generate a pkg we should migrate the environment over from Scons to Meson.

Perfect. We already have a ps3 package from a simple c++ sdl2

./easy-compile-docker-ps3 && ./release/release-ps3

image

@juvinious
Copy link
Collaborator

juvinious commented Feb 20, 2024

Ok got it to work. Had to change the files since the ones you uploaded were uppercase.
image
Also added an argument to allow for a different data directory.

Edit.
Though, the full data directory is put into the pkg. Is there a reason we get a red box? Doesn't that mean it can't find data?

@humbertodias
Copy link
Collaborator Author

Ok got it to work. Had to change the files since the ones you uploaded were uppercase. image Also added an argument to allow for a different data directory.

Edit. Though, the full data directory is put into the pkg. Is there a reason we get a red box? Doesn't that mean it can't find data?

Perfect! no problem.
Actually, the red box it's an example that I´m using instead of paintown project.
Because, I still didn't solve all dependencies on meson build.
Meanwhile, with this sample app. I have validated the ci/cd

@juvinious
Copy link
Collaborator

Ah ok, now it makes sense.

@humbertodias
Copy link
Collaborator Author

Ah ok, now it makes sense.

The error occurred while attempting to compile yaml-cpp using psp-g++
Same for ps3-dev

How to reproduce

docker run --platform=linux/amd64 --rm -i -v `mktemp -d`:/source pspdev/pspdev:latest sh -s <<EOF
wget https://github.com/jbeder/yaml-cpp/archive/refs/tags/0.8.0.tar.gz
tar xvfz 0.8.0.tar.gz
cd yaml-cpp-0.8.0
env
mkdir -p build 
cd build
cmake -Wno-dev -DCMAKE_TOOLCHAIN_FILE=/usr/local/pspdev/psp/share/pspdev.cmake \
-DCMAKE_INSTALL_PREFIX=/usr/local/pspdev/psp -DBUILD_SHARED_LIBS=OFF ..
make 
make install
EOF

Error

- Build files have been written to: /yaml-cpp-0.8.0/build
[  2%] Building CXX object CMakeFiles/yaml-cpp.dir/src/contrib/graphbuilder.cpp.obj
cc1plus: error: cannot generate position-independent code for ‘-mabi=eabi’
make[2]: *** [CMakeFiles/yaml-cpp.dir/build.make:76: CMakeFiles/yaml-cpp.dir/src/contrib/graphbuilder.cpp.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:861: CMakeFiles/yaml-cpp.dir/all] Error 2

@juvinious
Copy link
Collaborator

juvinious commented Feb 20, 2024

Easy solution is to not include yaml. It's not required.
Make sure -DHAVE_YAML is not set.

@kazzmir
Copy link
Owner

kazzmir commented Feb 20, 2024

The error suggests a general issue with compiling for a specific abi though, should we remove that -meabi flag?

@humbertodias
Copy link
Collaborator Author

The error suggests a general issue with compiling for a specific abi though, should we remove that -meabi flag?

Yeah. Even removing, fail. weird.

@humbertodias
Copy link
Collaborator Author

Easy solution is to not include yaml. It's not required. Make sure -DHAVE_YAML is not set.

i'm gonna try

@humbertodias
Copy link
Collaborator Author

Easy solution is to not include yaml. It's not required. Make sure -DHAVE_YAML is not set.

I found the problem and submitted a fix jbeder/yaml-cpp#1271

@juvinious
Copy link
Collaborator

Cool, hopefully they accept it but I bet they'll ask for something more generic (i.e. not ps3/psp) to check. But who knows.
What I did was just not include yaml support for now for the dkp build by checking if it was downloaded or not:

if meson.is_cross_build()
  fs = import('fs')
  if fs.is_dir('.tmp/yaml-cpp')
    add_global_arguments('-DHAVE_YAML_CPP', language: 'cpp')
    subdir('.tmp/yaml-cpp')
  endif
  subdir('.tmp/SDL_gfx')
endif

@kazzmir
Copy link
Owner

kazzmir commented Feb 21, 2024

We could apply a local patch to the yaml code, if upstream won't accept a change

@humbertodias
Copy link
Collaborator Author

We could apply a local patch to the yaml code, if upstream won't accept a change

Good idea!
I have overpass the yaml-cpp problem.
Now another with r-tech1/libgme
How to reproduce

git clone https://github.com/kazzmir/paintown -b build-psp
cd paintown
./easy-compile-docker-psp

Error
https://github.com/kazzmir/paintown/actions/runs/7994410565/job/21832385288?pr=93#step:3:10345

  User defined options
    Cross files: psp-cross.txt

Found ninja-1.11.1 at /usr/bin/ninja
(cd build-psp; meson configure -Dbuild_tests=false)
meson compile  -C build-psp
ninja: Entering directory `/paintown-bin/build-psp'
[1/352] Compiling C++ object src/r-tech1/libs/gme/libgme.a.p/Ay_Apu.cpp.o
[2/352] Compiling C++ object src/r-tech1/libs/gme/libgme.a.p/Data_Reader.cpp.o
[3/352] Compiling C++ object src/r-tech1/libs/gme/libgme.a.p/Ay_Cpu.cpp.o
FAILED: src/r-tech1/libs/gme/libgme.a.p/Ay_Cpu.cpp.o 
psp-g++ -Isrc/r-tech1/libs/gme/libgme.a.p -Isrc/r-tech1/libs/gme -I../src/r-tech1/libs/gme -Isrc -I../src -I/psp/include -I/pspdev/psp/include -I/pspdev/psp/sdk/include -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++11 -O0 -g -DUSE_SDL2=1 '-DDATA_PATH="data"' -g3 -Wextra -Wno-unused-variable -Wno-unused-function -Wno-unused-parameter -DHAVE_OGG=1 -DHAVE_MP3_MPG123=1 -DPSP -D_PSP_FW_VERSION=500 -w -MD -MQ src/r-tech1/libs/gme/libgme.a.p/Ay_Cpu.cpp.o -MF src/r-tech1/libs/gme/libgme.a.p/Ay_Cpu.cpp.o.d -o src/r-tech1/libs/gme/libgme.a.p/Ay_Cpu.cpp.o -c ../src/r-tech1/libs/gme/Ay_Cpu.cpp
../src/r-tech1/libs/gme/Ay_Cpu.cpp:104:10: error: #error "Byte order of CPU must be known"
  104 |         #error "Byte order of CPU must be known"
      |          ^~~~~
../src/r-tech1/libs/gme/Ay_Cpu.cpp: In member function ‘bool Ay_Cpu::run(cpu_time_t)’:
../src/r-tech1/libs/gme/Ay_Cpu.cpp:413:24: error: ‘R8’ was not declared in this scope; did you mean ‘RL’?
  413 |                 data = R8( opcode & 7, 0 );
      |                        ^~
      |                        RL
[4/352] Compiling C++ object src/r-tech1/libs/gme/libgme.a.p/Classic_Emu.cpp.o

@kazzmir
Copy link
Owner

kazzmir commented Feb 21, 2024

https://en.wikipedia.org/wiki/PlayStation_Portable_hardware

Looks like the PSP is little endian. We just have to define BLARGG_LITTLE_ENDIAN in gme/blargg_endian.h for the PSP

@kazzmir
Copy link
Owner

kazzmir commented Feb 21, 2024

Well this github issue is supposed to be about the ps3 build, but now we are discussing psp. Maybe we should move psp discussion to its own issue?

@humbertodias
Copy link
Collaborator Author

Well this github issue is supposed to be about the ps3 build, but now we are discussing psp. Maybe we should move psp discussion to its own issue?

Oh.. that's true. I'm gonna create another issue for psp

@humbertodias
Copy link
Collaborator Author

https://en.wikipedia.org/wiki/PlayStation_Portable_hardware

Looks like the PSP is little endian. We just have to define BLARGG_LITTLE_ENDIAN in gme/blargg_endian.h for the PSP

Fixed e0f5bcb

@humbertodias
Copy link
Collaborator Author

Cool, hopefully they accept it but I bet they'll ask for something more generic (i.e. not ps3/psp) to check. But who knows. What I did was just not include yaml support for now for the dkp build by checking if it was downloaded or not:

if meson.is_cross_build()
  fs = import('fs')
  if fs.is_dir('.tmp/yaml-cpp')
    add_global_arguments('-DHAVE_YAML_CPP', language: 'cpp')
    subdir('.tmp/yaml-cpp')
  endif
  subdir('.tmp/SDL_gfx')
endif

Merged jbeder/yaml-cpp#1271

@humbertodias
Copy link
Collaborator Author

Unfortunately the container PSL1GHT doesn't include SDL2_image|ttf|mixer|glx and I'm gonna add on ps3libraries

@humbertodias
Copy link
Collaborator Author

Unfortunately the container PSL1GHT doesn't include SDL2_image|ttf|mixer|glx and I'm gonna add on ps3libraries

Image ps3dev-sdl2 created

Content:

SDL_PSL1GHT + SDL_PSL1GHT_Libs

  • sdl - 1.3.0
  • sdl_mixer - 1.2.11
  • sdl_image - 1.2.10
  • sdl_ttf - 2.0.10
  • sdl_net - 1.2.7
  • sdl_gfx - 2.0.27

SDL2_PSL1GHT + SDL2_PSL1GHT_Libs

  • sdl2 - 2.0.13
  • sdl2_mixer - 2.8.0
  • sdl2_image - 2.8.2
  • sdl2_ttf - 2.22.0
  • sdl2_gfx - 2.22.0

@humbertodias
Copy link
Collaborator Author

humbertodias commented Mar 4, 2024

Are we going to use pthread instead of std:thread or the same approach used for file-system?

[143/361] Compiling C++ object src/r-tech1/libr-tech1.a.p/graphics_bitmap.cpp.o
FAILED: src/r-tech1/libr-tech1.a.p/graphics_bitmap.cpp.o 
ppu-g++ -Isrc/r-tech1/libr-tech1.a.p -Isrc/r-tech1 -I../paintown-bin/src/r-tech1 -Isrc -I../paintown-bin/src -I/usr/local/ps3dev/portlibs/ppu/include/SDL2 -I/usr/local/ps3dev/portlibs/ppu/include -I/usr/local/include -I/usr/local/ps3dev/ppu/include/ -I/usr/local/ps3dev/portlibs/ppu/include/ -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++11 -O0 -g -DUSE_SDL2=1 -g3 -Wextra -Wno-unused-variable -Wno-unused-function -Wno-unused-parameter -DCROSS_BUILD -DHAVE_OGG=1 -mhard-float -fmodulo-sched -ffunction-sections -fdata-sections -maltivec -mminimal-toc -DPS3 -DHAVE_YAML_CPP -mcpu=cell -fno-builtin -nostdlib '-DDATA_PATH="/dev_hdd0/game/paintown/data"' -fPIC -MD -MQ src/r-tech1/libr-tech1.a.p/graphics_bitmap.cpp.o -MF src/r-tech1/libr-tech1.a.p/graphics_bitmap.cpp.o.d -o src/r-tech1/libr-tech1.a.p/graphics_bitmap.cpp.o -c ../paintown-bin/src/r-tech1/graphics/bitmap.cpp
In file included from ../paintown-bin/src/r-tech1/file-system.h:6:0,
                 from ../paintown-bin/src/r-tech1/funcs.h:10,
                 from ../paintown-bin/src/r-tech1/graphics/bitmap.cpp:1:
../paintown-bin/src/r-tech1/thread.h:180:10: error: 'thread' in namespace 'std' does not name a type
     std::thread thread;
          ^~~~~~
../paintown-bin/src/r-tech1/thread.h:296:10: error: 'thread' in namespace 'std' does not name a type
     std::thread thread;
          ^~~~~~
../paintown-bin/src/r-tech1/thread.h: In destructor 'virtual Util::Future<X>::~Future()':
../paintown-bin/src/r-tech1/thread.h:219:13: error: 'thread' was not declared in this scope

https://github.com/kazzmir/paintown/actions/runs/8146492951/job/22265104133#step:3:404

@kazzmir
Copy link
Owner

kazzmir commented Mar 4, 2024

it would be nice if we could continue to use std::thread, and have std::thread use some native thread implementation, or perhaps use sdl2's threading system. if that doesn't work then maybe I will write some wrapper around std::thread that can use whatever the native threading system is for each port

@kazzmir
Copy link
Owner

kazzmir commented Mar 4, 2024

what version of g++ is ppu-g++ based off of? if ppu-g++ is something recent, like gcc 11 then I would expect std::thread to be there. if not, maybe ppu-g++ can be updated?

@humbertodias
Copy link
Collaborator Author

humbertodias commented Mar 4, 2024

what version of g++ is ppu-g++ based off of? if ppu-g++ is something recent, like gcc 11 then I would expect std::thread to be there. if not, maybe ppu-g++ can be updated?

7.2.0

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run -it hldtux/ps3dev-sdl2 bash -c "ppu-gcc --version"
ppu-gcc (GCC) 7.2.0

@humbertodias
Copy link
Collaborator Author

humbertodias commented Mar 4, 2024

what version of g++ is ppu-g++ based off of? if ppu-g++ is something recent, like gcc 11 then I would expect std::thread to be there. if not, maybe ppu-g++ can be updated?

We are using the toolchain https://github.com/ps3dev/ps3toolchain
I found out a newer version - 13.2.0-1
https://aur.archlinux.org/packages/ps3-ppu-gcc

it's adapted for archlinux
but, they have left the patch
image
I'm gonna build gcc 13.2 with that patch and later on try to compile paintown with std:thread

@humbertodias
Copy link
Collaborator Author

humbertodias commented Mar 6, 2024

PR - Upgrading version - ps3dev/ps3toolchain#121

  • GNU Binutils 2.42
  • ppu-gcc (GCC) 13.2.0
  • spu-gcc (GCC) 9.5.0

Now we can retry to compile paintown with c11 feature std::thread

@humbertodias
Copy link
Collaborator Author

Success!
After upgrading ppu-gcc to 13 we have support to std:thread

[353/361] Compiling C++ object src/openbor/libopenbor.a.p/display-character.cpp.o
[354/361] Compiling C++ object src/openbor/libopenbor.a.p/pack-reader.cpp.o
[355/361] Compiling C++ object paintown.p/src_globals.cpp.o
[356/361] Compiling C++ object paintown.p/src_factory_collector.cpp.o
[357/361] Compiling C++ object paintown.p/src_factory_font_render.cpp.o
[358/361] Compiling C++ object src/openbor/libopenbor.a.p/mod.cpp.o
[359/361] Linking static target src/openbor/libopenbor.a
[360/361] Compiling C++ object paintown.p/src_main-menu.cpp.o
[361/361] Linking target paintown

Now it's remaining only runtime issues
image

@kazzmir
Copy link
Owner

kazzmir commented Mar 6, 2024

Awesome! How far does the game get at runtime? Does the menu show up?

@humbertodias
Copy link
Collaborator Author

humbertodias commented Mar 7, 2024

I have anticipated myself.
Even with c11 enabled the use of thread and lambda are not compiling.
Here for example https://github.com/kazzmir/paintown/actions/runs/8181132738/job/22370408154?pr=92#step:3:318

../paintown-bin/src/r-tech1/sound/music.cpp: In constructor 'Music::Music(bool)':
../paintown-bin/src/r-tech1/sound/music.cpp:57:50: error: no matching function for call to 'std::thread::thread(void (&)(Music*), Music*)'
   57 |         musicThread = std::thread(playMusic, this);

Well, at least now thread are detected on std namespace
Any thoughts on what it might be?

@humbertodias
Copy link
Collaborator Author

humbertodias commented Mar 7, 2024

I think i understood what is going on.
Let me try to explain and if you are gonna say if it makes sense or not.

Paintown uses std::thread with c++11 that is Object-Oriented and pthread-emb library inside the container is compiled only in c as procedural at pthread-emb-ps3

I think the problem is that.
pthread-emb-ps3 should be Object-Oriented as well to match and also be compiled with g++ at 039-pthread-emb.sh

Another point is that ppu-g++ is compiled with --enable-threads at 002-gcc-newlib-PPU.sh

Testing inside the container

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker run -it hldtux/ps3dev-sdl2 bash

procedural.c

#include <stdio.h>
#include <pthread.h>
#include <unistd.h> // For sleep() function

void* thread_function(void* arg) {
    int thread_num = *((int*)arg);
    printf("Thread %d is running\n", thread_num);
    sleep(1); // Simulate some work
    printf("Thread %d finished\n", thread_num);
    return NULL;
}

int main() {
    pthread_t thread1, thread2;
    int thread1_num = 1, thread2_num = 2;

    // Create threads
    pthread_create(&thread1, NULL, thread_function, (void*)&thread1_num);
    pthread_create(&thread2, NULL, thread_function, (void*)&thread2_num);

    // Wait for threads to finish
    pthread_join(thread1, NULL);
    pthread_join(thread2, NULL);

    printf("Both threads have finished\n");

    return 0;
}
ppu-gcc -std=c11 procedural.c -o procedural `pkg-config --libs --cflags pthread`
file procedural
procedural: ELF 64-bit MSB executable, 64-bit PowerPC or cisco 7500, Power ELF V1 ABI, version 1 (SYSV), statically linked, with debug_info, not stripped

oo.cpp

#include <iostream>
#include <thread>

// Function to be executed by the thread
void threadFunction(int threadID) {
    std::cout << "Thread " << threadID << " is executing" << std::endl;
}

int main() {
    // Creating a thread and passing a function to execute
    std::thread t1(threadFunction, 1);

    // Joining the thread to the main thread
    t1.join();

    // Output a message from the main thread
    std::cout << "Main thread is executing" << std::endl;

    return 0;
}
ppu-g++ -std=c++11 oo.cpp -o oo `pkg-config --libs --cflags pthread`

Result

cc1plus: warning: command-line option '-std=c11' is valid for C/ObjC but not for C++
oo.cpp: In function 'int main()':
oo.cpp:11:37: error: no matching function for call to 'std::thread::thread(void (&)(int), int)'
   11 |     std::thread t1(threadFunction, 1);
      |                                     ^
In file included from /usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/thread:45,
                 from oo.cpp:2:
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:178:5: note: candidate: 'std::thread::thread(std::thread&&)'
  178 |     thread(thread&& __t) noexcept
      |     ^~~~~~
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:178:5: note:   candidate expects 1 argument, 2 provided
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:132:5: note: candidate: 'std::thread::thread()'
  132 |     thread() noexcept = default;
      |     ^~~~~~
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:132:5: note:   candidate expects 0 arguments, 2 provided

Using -pthead

ppu-g++ -std=c++11 oo.cpp -o oo -pthread
cc1plus: warning: command-line option '-std=c11' is valid for C/ObjC but not for C++

Using -fpthread

ppu-g++ -std=c++11 oo.cpp -o oo -fpthread
ppu-g++: error: unrecognized command-line option '-pthread'; did you mean '-fpthread'?
cc1plus: warning: command-line option '-std=c11' is valid for C/ObjC but not for C++
cc1plus: warning: command-line option '-fpthread' is valid for Modula-2 but not for C++
oo.cpp: In function 'int main()':
oo.cpp:11:37: error: no matching function for call to 'std::thread::thread(void (&)(int), int)'
   11 |     std::thread t1(threadFunction, 1);
      |                                     ^
In file included from /usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/thread:45,
                 from oo.cpp:2:
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:178:5: note: candidate: 'std::thread::thread(std::thread&&)'
  178 |     thread(thread&& __t) noexcept
      |     ^~~~~~
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:178:5: note:   candidate expects 1 argument, 2 provided
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:132:5: note: candidate: 'std::thread::thread()'
  132 |     thread() noexcept = default;
      |     ^~~~~~
/usr/local/ps3dev/ppu/powerpc64-ps3-elf/include/c++/13.2.0/bits/std_thread.h:132:5: note:   candidate expects 0 arguments, 2 provided

@kazzmir
Copy link
Owner

kazzmir commented Mar 7, 2024

If you are compiling c++ code try adding -std=c++11 instead of -std=c11. But essentially std::thread should be implemented on top of pthread.

@humbertodias
Copy link
Collaborator Author

If you are compiling c++ code try adding -std=c++11 instead of -std=c11. But essentially std::thread should be implemented on top of pthread.

I have solved using Util::Thread instead.
Now we have the compilation working again. But on runtime it freezes and got stuck on a black screen.
I'm almost sure that something is missing on ps3dev and I'm still investigating

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants