Add support for GopherJS on web

[kkoreilly]

Describe the feature

Using https://github.com/tinygo-org/tinygo for web could drastically reduce binary sizes, making it much more viable to use Cogent Core for websites. I tried to do this and ran into a bunch of issues with reflect slice headers and missing os functions, but we should look at this again and try to get it working at some point.

Relevant code

No response

[kkoreilly]

We could also look into things like wasm-opt and https://github.com/gopherjs/gopherjs to reduce web binary sizes.

[gedw99]

these 2 things are quite important for adoption.

Prior work where both go, tinygo and gophers compilers exist is Go-app
https://go-app.dev which can run server side and client side.

tinygo can take a golang wasm from 6 mb to about 2 MB I found in general.

Also its possible to run on Cloudflare as WASM for free IF you get your WASM small.
https://github.com/syumai/workers
Then you have a global fast and free way to run your backend and can load up your FrontEnd from Cloudflare R3 to the Users browser.

---

If someone is building an internal tool they dont care.
It its public then the SEO and Time to first interaction matter.

There are a few mitigations also:

Have a Markdown version of the App, with Hugo.

• Content is created a s markdown also by the system. a type of event sourcing pattern.
• Hugo renders it on the fly.
• Google SEO is directed to the www.example.com
• Humans are directed to the app.example.com
• Caddy Reverse proxy can easily do that redirect, along with a caddy Module to detect if its a Human or a Search Bot.

Have a long splash page:

• You can design the app as a 2 stage app.
• First stage loads a simple thing that is quick. Maybe using gophsJS
• Second stage load the full thing.

[kkoreilly]

Yep, I have used go-app before and our web loading mechanism is based on theirs. I definitely want to reduce binary sizes and loading times and improve SEO support, as all of those things are critical for a good user experience.

Again, as I stated above, tinygo has an incompatible reflect slice header type, which makes it untenable to use with multiple of our dependencies that we cannot easily control. See tinygo-org/tinygo#1284 and https://tinygo.org/docs/guides/compatibility/ for more information. Either we can try to convince tinygo to change their reflect slice header type for wasm, or we can try to convince our dependencies to update to the new unsafe slice type.

For SEO, we have issue #702. I am planning to implement SEO by doing a server-side pre-rendering step that builds an HTML tree representing the content which can be used for SEO purposes, similar to go-app.

I already did work on making the loading screen look nicer, but we can certainly work more on having some sort of interactive first stage. Hopefully the loading time is fast enough with tinygo and other binary size optimizations that we don't need to worry too much about that.

[gedw99]

Hey @kkoreilly

thanks for the in depth answers.

For the tinygo issue, i suggest raising it with https://github.com/dgryski who is one of the maintainers. He helped me with a similar tinygo issue I had.

For SEO, HTML Server side rendering would be amazing. I presume that means for every widget, you're going to need a html equivalent ?

[kkoreilly]

I think it is unlikely that they will be receptive to this issue given that they already discussed it a lot in the issue I linked above and other issues, and decided to keep their incompatible reflect slice header type and advise people to use the compatible unsafe slice type instead. Therefore, I think our best chance is just filing issues and/or PRs for our dependencies that still use the incompatible type. I hope to be able to work on that somewhat soon, although various other issues may be higher priority.

Yes, we will need some HTML encoding for every widget. However, it hopefully shouldn't be too hard if we use a general xml marshaller in our base tree package and then implement a best-effort matching of our widget names and properties to corresponding HTML ones. Again, I am prioritizing other things that are more urgent, but I would like to work on this relatively soon.

[MatejMagat305]

please slow...., tinygo is wonderfull, but it is still 1 core program ..., multicore is now planing, but it is still in progres tinygo-org/tinygo#4047

[kkoreilly]

We are only talking about using tinygo for wasm. Go already does not support multicore on wasm, and neither does tinygo, so using tinygo should have no impact on that. See tinygo-org/tinygo#2630.

[kkoreilly]

I tried again to implement tinygo support, and I was able to get the basic example building after changing various things and commenting out all of the icons. It was 15 MB compared to 32 MB for the standard Go compilation, but when compressed, it was 5 MB, which is only slightly smaller than the compressed standard one, which is 6.4 MB. Therefore, given the various issues and limitations of tinygo and the long compile time, I think that we are unlikely to pursue this route for the time being.

[gedw99]

Wow this is kind of huge. I appreciate your work on getting it this far .

I think it’s worth not writing it off for ever . My 2 cents.. I agree there are road blocks, but the following may be pertinent in the decision matrix:

The binary size is not everything to pursue . The runtime perf is also worth checking , I feel. It might be surprising what the results are . Do you test runtime perf ? Actually is there a tester ? Gio headless with a timer and an api to reach the headless has been pretty useful for me . Regression tests become very easy .

long time compile will likely be addressed. I helped with that. It will play out as: get it working, then optimise the compile speed.

Also in case this helps even though it’s not relevant to gui WASM:
tinygo wasip2 is now in dev also.
Wazero are not going to support it until wasi ABI settles down. I don’t blame them but it’s just a matter of things stabilising.

if there is a branch to look at I would appreciate it. There might be things I can suggest

[kkoreilly]

Yes, we are definitely still planning to support tinygo at some point once these issues are resolved, it is in a better state, and we have more time. I am just saying that it will not be a priority for now due to the marginal gains and the various issues.

You can look at the tinygo branch if you want; it is mainly just temporarily commenting things out so that it builds. In addition to what is committed there, you also have to replace //go:embed with // //go:embed in icons/icongen.go because of tinygo-org/tinygo#4348, which is another reason that I am not particularly excited to implement tinygo support for now.

After you do all of that, examples/basic should build, although more complicated examples such as docs result in a crash in the tinygo interpreter for some reason.

But again, I am definitely still excited to implement tinygo support later after we have finished the initial release and some of these tinygo issues are resolved.

[gedw99]

your right. Get it out the door and swing back on tinygo.

ship ship ship :)

[kkoreilly]

We are now considering gopherjs to be the better solution for this. This is because tinygo only gives marginal binary size gains when compressed, whereas gopherjs appears to give 3x reductions in size when compressed. Also, gopherjs may bypass various other wasm performance issues. Here are the sizes of the a standard wasm output and a gopherjs output for a simple markdown example:

-rw-r--r--   1 kaioreilly  staff   1.5M Jul 24 19:49 debug.js
-rw-r--r--   1 kaioreilly  staff   309K Jul 24 19:49 debug.js.gz

-rwxr-xr-x   1 kaioreilly  staff   3.1M Jul 24 19:49 app.wasm
-rwxr-xr-x   1 kaioreilly  staff   902K Jul 24 19:49 app.wasm.gz

GopherJS still needs various improvements to work for Cogent Core, which we are planning to make soon (see gopherjs/gopherjs#1334).

[kkoreilly]

Another reason to use gopherjs is that it has much faster compile times than tinygo, and it can be installed directly using Go on any platform; it is much more in line with Go itself than tinygo.

[gedw99]

II agree. As much as I want to love WASM and golang, it's just not quite right.

https://github.com/maxence-charriere/go-app is a good example. they use WASM, but there was a big debate about also supporting gophers because of this WASM problem.

It's worth looking at this: maxence-charriere/go-app#830 where a PR to support gophers happened but never went through.

https://github.com/gopherjs/gopherjs has only 1 maintainer, but I think he is quite dedicated too. It worth reaching out to him perhaps to discuss things as he has excellent gophers experience and is very easy to work with from what I can see.

[kkoreilly]

Yep, I had looked at that go-app PR when I was using go-app. We already use js for the build tags (because I remembered that PR), so once we get GopherJS Go 1.22 support working, everything should be pretty good. We already reached out to the GopherJS peope (see gopherjs/gopherjs#1334).

[gedw99]

I use this to get around the many golang compilers and the fact that gophers does not support the latest. It leaves your normal golang env completely alone, which I like.

Even the makefile uses the main golang compiler.

Its very easy to write this same logic into your golang based build script, so then for developers they just never have to worry about golang versions that they have on their machine.

# os
BASE_OS_NAME := $(shell go env GOOS)
BASE_OS_ARCH := $(shell go env GOARCH)

GOBREW=gobrew
ifeq ($(BASE_OS_NAME),windows)
	GOBREW=gobrew.exe
endif
GOBREW_WHICH=$(shell command -v $(GOBREW))

# You can also do this. works on all OS's
export PATH:=$(PATH):$(HOME)/.gobrew/current/bin:$(HOME)/.gobrew/bin

print:
	@echo "GOBREW: $(GOBREW_WHICH)"

dep:
	# gobrew ( https://github.com/kevincobain2000/gobrew ) 
	go install github.com/kevincobain2000/gobrew/cmd/gobrew@latest
dep-del:
	rm -rf $(GOBREW_WHICH)

init:
	# For golang project the correct golang compiler version will be used.
	cd go-app && $(GOBREW) use mod
	cd lofimusic && $(GOBREW) use mod

[kkoreilly]

Yes, I know that we can get an old version of Go, but we need the features added in recent versions of Go, so we need to add support for them to GopherJS.

[gedw99]

I know. this makes it easier maybe for getting there I find.

[gedw99]

This means that there will be none of the WASM rendering to a WebGL canvas , but instead rendering to html for web, desktop and mobile ?

[kkoreilly]

No, we will still be rendering to a canvas, just using GopherJS. We will implement server-side pre-rendering for SEO in #702, and we are also planning to implement WebGPU support in #507, which will allow for 3D and hardware acceleration on the web.

[gedw99]

OK makes sense now.

Because you're trying to cover so many output mediums, a DSL might be warranted. The DSL is used by all the "render engines". Loopback for buttons or any forms etc. It's almost like a benthos stream processor running inline. It can be fast by just inlining it all.

[kkoreilly]

After we implement GopherJS and WebGPU support, we will see what the performance situation is like and go from there.