Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ?at=DATE flag #1051

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

Add ?at=DATE flag #1051

wants to merge 3 commits into from

Conversation

ije
Copy link
Member

@ije ije commented Jan 23, 2025

This pr allows uou to fetch a package version closest to a given timestamp or date. This feature eliminates the need to know the exact version beforehand, similar to running npm install <package> without specifying a version. However, it still provides a fixed version for future installations, ensuring consistency. This is particularly useful for LLM-generated web applications, where maintaining the same package version across all instances is crucial.

For example:

import React from "https://esm.sh/react?at=YYYY-MM-DD";
import React from "https://esm.sh/react?at=TIMESTAMP";

implements #1047

@sickl8
Copy link

sickl8 commented Jan 23, 2025

@ije hello again, I've read a lot of your code and implemented only the parsing part of the feature on my fork, can you go over how the tool works internally so that we can optimize the use of cache for the users utilizing the at feature?
my current assumption is you have some sort of custom hand-made redis cache for versions and entries have a certain TTL before being cleared from RAM, but I still haven't got my head around the DB and the .esmd/npm and when you use each of them.

@sickl8
Copy link

sickl8 commented Jan 23, 2025

https://github.com/sickl8/esm.sh/tree/feat-get-version-closest-to-date-or-timestamp

@ije
Copy link
Member Author

ije commented Jan 23, 2025

@ije
Copy link
Member Author

ije commented Jan 23, 2025

the server uses memory for caching npm package info with configurable TTL, no back storage

esm.sh/server/npm.go

Lines 347 to 352 in 02cb1c5

getCacheKey := func(pkgName string, pkgVersion string) string {
return reg.Registry + pkgName + "@" + pkgVersion
}
version = normalizePackageVersion(version)
return withCache(getCacheKey(pkgName, version), time.Duration(config.NpmQueryCacheTTL)*time.Second, func() (*PackageJSON, string, error) {

@ije
Copy link
Member Author

ije commented Jan 23, 2025

i just view your code, looks great. maybe you can start with this branch: https://github.com/esm-dev/esm.sh/tree/at-flag, you can get the at param form BuildContext.args.at, it's stored as a timestamp in uint32

@sickl8
Copy link

sickl8 commented Jan 23, 2025

i just view your code, looks great. maybe you can start with this branch: https://github.com/esm-dev/esm.sh/tree/at-flag, you can get the at param form BuildContext.args.at, it's stored as a timestamp in uint32

I think we should use an int64 or uint64 to align better with the time.Unix() function for the timestamp, do you have discord so we can discuss this further?

@ije
Copy link
Member Author

ije commented Jan 23, 2025

please join esm.sh server here: https://discord.gg/XDbjMeb7pb

@ije
Copy link
Member Author

ije commented Jan 23, 2025

i prefer using uint32 that is enough for unix time without nsec right? and it can be encode to 4 bytes (i want to keep the url as short as possible)

@sickl8
Copy link

sickl8 commented Jan 23, 2025

yes, but if the feature is going to support ms, the unit of the timestamp returned from javascript's Date.now() used for convenience, you would probably prefer using an uint64 to make life easier when parsing and using time.Unix(), but I'm down with whatever works for you

@sickl8
Copy link

sickl8 commented Jan 25, 2025

@ije you can check out my latest commit to the branch, I think I still have to update the README.md to document the feature, and also maybe add tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants