Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pure-go Sqlite #1925

Open
aymanbagabas opened this issue Jun 16, 2023 · 19 comments · May be fixed by #3051
Open

Pure-go Sqlite #1925

aymanbagabas opened this issue Jun 16, 2023 · 19 comments · May be fixed by #3051
Assignees
Labels

Comments

@aymanbagabas
Copy link

Hi all 👋

Couldn't help but notice that Miniflux doesn't support Sqlite because the driver needs CGo and GCC. Would like to share that there is a pure-go Sqlite driver that doesn't need CGo nor GCC https://gitlab.com/cznic/sqlite would be great if Miniflux supports Sqlite as its a lot lighter that Postgres.

@hauleth
Copy link

hauleth commented Dec 1, 2023

That would also make backing everything up way easier.

@gergan
Copy link
Contributor

gergan commented Jan 3, 2024

Hi @fguillot - are you open for a pull request adding sqlite support?

@fhemberger
Copy link

fhemberger commented Jan 3, 2024

@gergan I took a very quick peek at the code the other day. Seems you'd need to borrow two methods from the pq package, which are used in Miniflux:

Couldn't find any other code specificly tied to Postgres at first glance. So it looks like changing database drivers could be feasible without too much of a hassle.

@fguillot
Copy link
Member

fguillot commented Jan 4, 2024

Hi @fguillot - are you open for a pull request adding sqlite support?

I'm not against that. However, I'm a little bit concerned about the number of changes required.

Not sure either about the maturity of pure Go Sqlite drivers compared to https://github.com/mattn/go-sqlite3

@gergan
Copy link
Contributor

gergan commented Jan 10, 2024

I've had a quick look at the table definitions (DDL) and there will be some problems - for example the hstore extension, there is no "with timezone" datetime type in sqlite. Probably there will be tons of other problems too. I will not have the time in January to do any work on this.
Probably the best way would be to just try to make the code work with sqlite and if the changes to the code are not minimal, than probably it is not a good idea.

@hauleth
Copy link

hauleth commented Jan 11, 2024

It is weird that Miniflux uses timestamp with timezone at all, as this type is in general discouraged if you are using any form of ORM or anything. It has only a little bit sense when there is human operator on the DB, but other than that it introduces more problems than it solves. And hstore technically can be replaced by JSON fields.

@E021ntox
Copy link

E021ntox commented Jul 22, 2024

sqlite, please. postgresql is the biggest weakness of miniflux.

@pentago
Copy link

pentago commented Aug 27, 2024

Id love sqlite too.

@clarfonthey
Copy link
Contributor

Just poking in here: I know that GoToSocial also had to look for a pure-Go binding to SQLite, and this is the one they use: https://gitlab.com/cznic/sqlite

Probably worth checking out.

@gergan
Copy link
Contributor

gergan commented Aug 30, 2024

There are multiple implementations of Go SQLite without CGo - SQLite Benchmarks. The one that you mean (@clarfonthey) is probably the modernc.org one in the benchmark. I have looked over the database once more yesterday, and the hstore field was converted to separate fields in the meantime. But there are many timestamp with time zone fields there, and I'm not sure why they need to be in local time zones. As @hauleth already said, this probably has some historical reasons - the first step would probably be to try and convert the complete database to UTC. And after that, look at what still needs to be done for SQLite.

@jbrodriguez
Copy link

hope this gains some traction, id love to go back to a lightweight version of google reader

@jbrodriguez
Copy link

i found some additional info about timestamp with time zone (timestamptz) in this article https://community.spiceworks.com/topic/2454825-zone-of-misunderstanding

what i understand is that the postgres driver performs the time conversion from db stored time with timezone to the local time of the user hitting the db, and this is something that postgres guarantees (or is it a guarantee that the client lib is enforcing? 🤷‍♂️)

using sqlite, this would have to be done in client code

in any case, timezones are hard 😀

p.s.: the postgres article where i found the link to the timestamp with time zone reference https://challahscript.com/what_i_wish_someone_told_me_about_postgres?ref=labnotes.org#follow-all-the-advice-from-the-folks-that-make-postgres

p.s.: i tested freshrss as it uses sqlite by default, but it just didn't work, nowhere near as clean and no-nonsense as miniflux

@dertuxmalwieder
Copy link
Contributor

As much as I admire @jvoisin's work on this, I think the Miniflux FAQ provides excellent arguments against SQLite. I would find it somewhat regrettable if this clear stance were to be abandoned.

@onedr0p
Copy link

onedr0p commented Jan 15, 2025

@dertuxmalwieder keep in mind the first point in the link you shared.

Focus on Simplicity

Sqlite makes miniflux simpler for users to deploy and maintain since they don't need to add more complexity with an external database. From a code standpoint sure supporting multiple DBs can add complexity for the developers but that's their decision to make if they find value in it.

provides excellent arguments against SQLite

Other than what I wrote above, the link you shared only mentions CGO as a major blocker for adding sqlite support which now can be avoided.

@jbrodriguez
Copy link

postgresql is an outstanding db, it powers fortune 500 companies, however, as others mentioned, it does add complexitiy in the miniflux use case, there may be tradeoffs in features, but im not really sure they're essential/critical 🤷‍♂️

@jasonbraganza
Copy link

jasonbraganza commented Jan 15, 2025

… I think the Miniflux FAQ provides excellent arguments against SQLite. I would find it somewhat regrettable if this clear stance were to be abandoned.

If anything, @jvoisin’s efforts would make things simpler for most of us.
I have been using Miniflux for close to three years now and the only knock I have against it, is its use of Postgres.

I for one am going to be amongst the first to switch to Sqlite once it's supported

@dertuxmalwieder
Copy link
Contributor

most of us.

[Citation needed]. PostgreSQL is not an obscure niche application on web-facing servers. SQLite is.

@hauleth
Copy link

hauleth commented Jan 15, 2025

SQLite isn't that obscure in case of self-hosting applications. A lot of them support SQLite as it greatly simplifies self-hosting (wrt. backups, management, securing stuff, etc.).

@jasonbraganza
Copy link

most of us.

[Citation needed]. PostgreSQL is not an obscure niche application on web-facing servers. SQLite is.

Neither are obscure and SQLite’s popularity for the kind of audience Miniflux has, is not in doubt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging a pull request may close this issue.