Added Solid support #1319

yasharpm · 2024-08-29T10:26:51Z

Hello, I have implemented the Solid backend. These changes are compatible with the widget. I am creating another pull request for the widget as well.

The test code and documentation updates are also there...

Thank you :-)

…e.js

…ge.js into merger

…ge.js

yasharpm · 2024-08-29T10:29:32Z

remotestorage/remotestorage-widget#136

raucao · 2024-08-29T11:23:06Z

Great to see a PR for another open protocol!

Feel free to also notify people on the community forums about these PRs: https://community.remotestorage.io/t/adding-solid-as-a-backend/828

Maybe you could also add some information for people new to SOLID (like myself e.g.) about how to test this end-to-end (i.e. which server implementation and.or popular provider/hoster to use).

DougReeder

This is wonderful; thank you!

A big change like this will take us a while to review and may take a couple rounds of changes (hopefully small); please be patient.

~~1. Have you had a chance to run the API test suite against this? rs-backup tweaked for your server is the easiest way to generate a token, that I've found.~~

Do you know if the Solid library is using fetch or XHR? We've been supporting environments with either one; if Solid only uses fetch we'll need to document that.
I see you've written tests using Jaribu. Unfortunately, that has been deprecated and doesn't run under recent versions of Node.js. New tests are being written using Mocha. I know it's a pain, but could you please add Mocha tests for Solid?

DougReeder · 2024-08-29T13:27:13Z

src/interfaces/configObserver.ts

Can you please add some comment lines, explaining how this file fits into the architecture?

If this interface is only used for Solid code, then you can move it to the Solid module and export it from there.

DougReeder · 2024-08-29T13:29:18Z

src/remotestorage.ts

@@ -368,7 +370,7 @@ export class RemoteStorage {
    if (typeof cfg === 'object') { extend(config, cfg); }

    this.addEvents([
-      'ready', 'authing', 'connecting', 'connected', 'disconnected',
+      'ready', 'authing', 'connecting', 'pod-not-selected', 'connected', 'disconnected',


I can imagine some reasons for 'pod-not-selected' to be separate from 'error', but please detail them.

... because no existing app will respond to the 'pod-not-selected' event, but they will respond to other events.

This would be the state of the Solid connection process for the widget or someone implementing custom UI. So it should be part of a documentation page for adding Solid to an app (maybe another section on the GDrive/Dropbox page, and renaming the page and menu item).

raucao · 2024-08-29T20:28:57Z

Have you had a chance to run the API test suite against this?

The API test suite is irrelevant here, as it tests the compliance of RS servers with the RS specification. However, this PR is for clients connecting to Solid servers, as an alternative to RS servers.

raucao · 2024-08-29T20:32:20Z

Another note for tests: there is a new Mocha suite for the RemoteStorage class in this open PR: #1318

DougReeder · 2024-08-29T20:35:30Z

Have you had a chance to run the API test suite against this?

The API test suite is irrelevant here, as it tests the compliance of RS servers with the RS specification. However, this PR is for clients connecting to Solid servers, as an alternative to RS servers.

Right you are! I've been wrestling with Armadietto for half a year. :-S

yasharpm · 2024-09-02T13:37:44Z

I've updated the documentation and added mocha tests. This should be it as far as I followed the conversation. What do you guys think?

DougReeder · 2024-09-04T13:50:32Z

I haven't been able to use this in a real app yet — the app I've been testing with is throwing up issues that don't appear to be caused by this. I'll have to try with a different app.

NoelDeMartin · 2024-09-12T00:23:38Z

Hi there, I don't know much about remoteStorage; but I've been working on Solid Apps and something crucial to Solid is using RDF for interoperability. Seeing this PR, I understand that this adds support for a Solid backend as a "file storage", right?

I wonder how would anyone use this library to make a Solid application using RDF. Would the developers need to write RDF manually? What about updates to documents?

Anyways, I'm just mentioning this in case you weren't aware. If that's already the intention, using Solid as a file storage rather than a store of RDF documents, that's fine. But I think using a Solid POD like that is missing a lot of the potential. I guess it's better than nothing though, if it adds "Solid support" to some of the existing apps implemented using remoteStorage. But I don't think I'd recommend anyone making a Solid App from scratch to follow this approach.

By the way, in case it's useful for anyone, some years ago we had a discussion comparing remoteStorage, Fission, and Solid. And this was one of the points that came up: https://vimeo.com/779640162

DougReeder · 2024-09-12T02:36:31Z

Anyways, I'm just mentioning this in case you weren't aware. If that's already the intention, using Solid as a file storage rather than a store of RDF documents, that's fine. But I think using a Solid POD like that is missing a lot of the potential. I guess it's better than nothing though, if it adds "Solid support" to some of the existing apps implemented using remoteStorage. But I don't think I'd recommend anyone making a Solid App from scratch to follow this approach.

Yes, you won't get many of the benefits of a native Solid app. But this is an important step in moving people to storage independent of apps. With this, someone who has a Solid pod can also use the same storage for Apps written for remoteStorage.

raucao · 2024-09-12T11:36:55Z

Wouldn't that be something that could be supported in the data modules for this library?

yasharpm · 2024-09-12T12:07:49Z

Wouldn't that be something that could be supported in the data modules for this library?

This was the idea when we were considering the task. The data module receives the Client instance which probably exposes the RemoteStorage hence the Solid backend which then you can get the raw Solid Session from it.

So a data module can implement a specific behavior for dealing with Solid the way it is recommended. The downside is that in order to be Solid friendly is that the different data modules need to have a check for the back-end type and do different thins.

DougReeder · 2024-09-13T22:02:59Z

I've tried adding this and the updated widget to a simpler app, but no option for Solid storage appears in the list. Is there an example of this running in a real app?

…ordering of devDependencies

michielbdejong · 2024-12-11T14:00:16Z

I'm trying this out in remotestorage/myfavoritedrinks#15

michielbdejong · 2024-12-11T15:37:33Z

There seems to be a bug related to folder listing. Storing 'beer' as my favorite drink works, but then it repeats GET 'https://michielbdejong.solidcommunity.net/myfavoritedrinks/' -H 'accept: text/turtle' in a loop and 'synchronizing' spins indefinitely. This is the response body. Is it supposed to fetch Turtle, or was the intention to fetch the folder listing as JSON-LD?

@prefix : <#>.
@prefix dct: <http://purl.org/dc/terms/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix stat: <http://www.w3.org/ns/posix/stat#>.
@prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
@prefix myf: <>.
@prefix json: <http://www.w3.org/ns/iana/media-types/application/json#>.

myf:
    a ldp:BasicContainer, ldp:Container;
    dct:modified "2024-12-11T13:42:10Z"^^xsd:dateTime;
    ldp:contains myf:beer;
    stat:mtime 1733924530.183;
    stat:size 4096 .
myf:beer
    a json:Resource, ldp:Resource;
    dct:modified "2024-12-11T13:42:10Z"^^xsd:dateTime;
    stat:mtime 1733924530.183;
    stat:size 88 .

michielbdejong · 2024-12-12T12:27:22Z

I found a bug in this PR. The code that produces folder item maps is not including ETags (lines 414 and 419 of src/solid.ts)
But sync considers an items map without ETags as corrupt. If the server doesn't give ETag headers, I think we can just use the last-modified dates as etags.

michielbdejong · 2025-02-04T15:33:21Z

@DougReeder

Is there an example of this running in a real app?

That was a good question! With a few changes I made, it's working in remotestorage/myfavoritedrinks#15 now.

So what should be our next step?

DougReeder

The code looks fine and passed the automated tests when I checked it. Now that we have confirmation that it works in a real app, all we need is documentation of 'pod-not-selected', and anything else that a app programmer needs to know when integrating this. Then we can merge!

michielbdejong · 2025-02-17T14:54:27Z

I'll get back to this in May in preparation for https://www.localfirstconf.com/ where I can hopefully give a presentation about it
I also mentioned it in https://www.youtube.com/watch?v=i66WxqUz0u8&t=1990s with a shout-out to @yasharpm for this awesome work :)

yasharpm and others added 23 commits October 24, 2023 23:11

Developing Solid storage

fa80bb4

Documentation for Solid

3a7eb40

Refactored Solid provider specification

3abd9a1

Login process completed. Next it choosing a pod

128728e

Login done (without error handling)

710d302

Fixed state management between rs and inrupt

c1b14bc

Get, Put, Delete implemented

aea6948

Kickstart workflows!

2294ae5

Kickstart workflows

3b3d4f9

add: workflow dispatch for test-and-lint.yml

2a7f9fe

Clean up

9fe0db1

Merge branch 'master' of https://github.com/pondersource/remotestorag…

6075b69

…e.js

Merged with upstream

818f0ce

Merge branch 'master' of https://github.com/remotestorage/remotestora…

555b565

…ge.js into merger

Base tests for solid

4a55b71

Tests done for solid backend

07bcf1c

Fixed lint issues

8facb19

Fixed TODOs

c11defb

merged with upstream

495baab

Added code documentation

8273b92

Updated readme

e6e92e0

Updated docs

ba000bc

Merge branch 'master' of https://github.com/remotestorage/remotestora…

e813dbb

…ge.js

yasharpm mentioned this pull request Aug 29, 2024

Added Solid storage support remotestorage/remotestorage-widget#136

Open

DougReeder requested changes Aug 29, 2024

View reviewed changes

yasharpm added 2 commits August 30, 2024 23:10

Updated documentation

afdfcf9

Mocha tests for Solid backend

f6a95f6

yasharpm requested review from DougReeder and raucao September 3, 2024 12:07

michielbdejong mentioned this pull request Dec 11, 2024

2024-50 dev log michielbdejong/devlog#9

Closed

michielbdejong added 4 commits December 11, 2024 14:05

Merge branch 'master' into HEAD

04c5720

Allow 'solid' as a value for backendType

5f168a8

Resolve peer dependency version mismatch for typedoc, fix alphabetic …

869a0f7

…ordering of devDependencies

npm install ; npm run build:release

89f1655

michielbdejong added a commit to remotestorage/myfavoritedrinks that referenced this pull request Dec 11, 2024

Use rs-js from remotestorage/remotestorage.js#1319

aa2ee9b

michielbdejong added 2 commits December 12, 2024 12:28

Console.log statements; this.getPodURLs() might be null

bfa89a1

getPodURLs may return null

c15323f

michielbdejong added 3 commits December 12, 2024 14:02

Use mtime as ETag; convert response Blob to text

7972cc9

remove console.log statements

fb82cd4

npm run build:release

fbc3bfc

This was referenced Feb 4, 2025

TUBS basics for Solid Slack Bridge michielbdejong/devlog#16

Open

2025-06 dev log michielbdejong/devlog#17

Closed

DougReeder requested changes Feb 10, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added Solid support #1319

Added Solid support #1319

yasharpm commented Aug 29, 2024

yasharpm commented Aug 29, 2024

raucao commented Aug 29, 2024

DougReeder left a comment •

edited

Loading

DougReeder Aug 29, 2024

yasharpm Aug 30, 2024

raucao Aug 31, 2024

DougReeder Aug 29, 2024

DougReeder Aug 29, 2024

raucao Aug 29, 2024 •

edited

Loading

raucao commented Aug 29, 2024 •

edited

Loading

raucao commented Aug 29, 2024

DougReeder commented Aug 29, 2024

yasharpm commented Sep 2, 2024 •

edited

Loading

DougReeder commented Sep 4, 2024

NoelDeMartin commented Sep 12, 2024

DougReeder commented Sep 12, 2024

raucao commented Sep 12, 2024

yasharpm commented Sep 12, 2024

DougReeder commented Sep 13, 2024

michielbdejong commented Dec 11, 2024

michielbdejong commented Dec 11, 2024

michielbdejong commented Dec 12, 2024

michielbdejong commented Feb 4, 2025 •

edited

Loading

DougReeder left a comment

michielbdejong commented Feb 17, 2025

Added Solid support #1319

Are you sure you want to change the base?

Added Solid support #1319

Conversation

yasharpm commented Aug 29, 2024

yasharpm commented Aug 29, 2024

raucao commented Aug 29, 2024

DougReeder left a comment • edited Loading

Choose a reason for hiding this comment

DougReeder Aug 29, 2024

Choose a reason for hiding this comment

yasharpm Aug 30, 2024

Choose a reason for hiding this comment

raucao Aug 31, 2024

Choose a reason for hiding this comment

DougReeder Aug 29, 2024

Choose a reason for hiding this comment

DougReeder Aug 29, 2024

Choose a reason for hiding this comment

raucao Aug 29, 2024 • edited Loading

Choose a reason for hiding this comment

raucao commented Aug 29, 2024 • edited Loading

raucao commented Aug 29, 2024

DougReeder commented Aug 29, 2024

yasharpm commented Sep 2, 2024 • edited Loading

DougReeder commented Sep 4, 2024

NoelDeMartin commented Sep 12, 2024

DougReeder commented Sep 12, 2024

raucao commented Sep 12, 2024

yasharpm commented Sep 12, 2024

DougReeder commented Sep 13, 2024

michielbdejong commented Dec 11, 2024

michielbdejong commented Dec 11, 2024

michielbdejong commented Dec 12, 2024

michielbdejong commented Feb 4, 2025 • edited Loading

DougReeder left a comment

Choose a reason for hiding this comment

michielbdejong commented Feb 17, 2025

DougReeder left a comment •

edited

Loading

raucao Aug 29, 2024 •

edited

Loading

raucao commented Aug 29, 2024 •

edited

Loading

yasharpm commented Sep 2, 2024 •

edited

Loading

michielbdejong commented Feb 4, 2025 •

edited

Loading