Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to tools.deps #872

Merged
merged 13 commits into from
Aug 20, 2023
22 changes: 10 additions & 12 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ version: 2.0
jobs:
build:
docker:
- image: clojure:openjdk-17-lein-buster
- image: clojure:temurin-17-tools-deps-bullseye
environment:
- DB_PORT: 5432
- CLOJARS_ENVIRONMENT: test
Expand All @@ -21,25 +21,23 @@ jobs:
- checkout
- restore_cache:
keys:
- clojars-{{ checksum "project.clj" }}
- run: lein deps
- clojars-{{ checksum "deps.edn" }}
- run: make prep-deps
- save_cache:
paths:
- $HOME/.m2
- $HOME/.lein
key: clojars-{{ checksum "project.clj" }}
key: clojars-{{ checksum "deps.edn" }}
# Install some tooling we need that isn't on the image by default
- run: apt update && apt install curl --yes && apt install unzip --yes
- run:
name: Download ElasticMQ
command: wget https://s3-eu-west-1.amazonaws.com/softwaremill-public/elasticmq-server-1.4.2.jar -O elasticmq.jar
command: curl https://s3-eu-west-1.amazonaws.com/softwaremill-public/elasticmq-server-1.4.2.jar -o elasticmq.jar
- run:
name: Run ElasticMQ
command: java -Dconfig.file=dev-resources/elasticmq.conf -jar elasticmq.jar
background: true
- run: lein do run -m user/migrate, test, uberjar
- run: ./bin/clj-kondo --lint src test
- run: make migrate-db
- run: make lint
- run: make test
- store_test_results:
path: target/test-results
- run: mv target/uberjar/clojars-web-*-standalone.jar clojars-uberjar.jar
- store_artifacts:
path: clojars-uberjar.jar
destination: uberjar
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,4 @@ pom.xml

.clj-kondo/.cache
bin/.cache
.cpcache
39 changes: 39 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
.PHONY: check
check:
clojure -M:defaults:check

.PHONY: lint
lint:
./bin/clj-kondo --lint src test

.PHONY: migrate-db
migrate-db:
clojure -M:defaults:migrate-db

.PHONY: nvd-check
nvd-check:
./bin/nvd-check

.PHONY: prep-deps
prep-deps:
clojure -A:defaults -X:deps prep

.PHONY: repl
repl:
clj -A:defaults:dev

.PHONY: setup-dev-repo
setup-dev-repo:
clojure -M:defaults:setup-dev-repo

.PHONY: tag-release
tag-release:
clj -T:build tag-release

.PHONY: test
test:
./bin/kaocha

.PHONY: uberjar
uberjar:
clojure -T:build uberjar
21 changes: 8 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,14 @@ website.
Development
-----------

**Note: Java 17 is required**
**Note: Java 17 and `make` are required**

### Development system

To begin developing, start with a REPL.
To begin developing, start with a REPL (Note: if you are instead starting a repl from your editor, you will need to include the `:default` and `:dev` aliases).

```sh
lein repl
make repl
```

You'll need elasticmq, minio, and postgres running as well. That's managed via
Expand All @@ -62,7 +62,7 @@ user=> (migrate)
or alternatively, from the command line.

```sh
$ lein migrate
$ make migrate-db
```

Run `go` to initiate and start the system.
Expand Down Expand Up @@ -99,7 +99,7 @@ setup a search index:
```sh
mkdir data/dev_repo
cp -r ~/.m2/repository/* data/dev_repo
lein run -m clojars.tools.setup-dev
make setup-dev-repo
```

Note that this setup task isn't perfect - SNAPSHOTS won't have
Expand All @@ -122,17 +122,12 @@ user=> (test #'clojars.test.unit.db/added-users-can-be-found)
...
```

Tests can also be run through Leiningen for CI.
Tests can also be run via [kaocha](https://github.com/lambdaisland/kaocha):

```sh
lein test
make test
```

### Production system with development config

Occasionally it can be useful to start a production system based on the development
configuration. This can be done by `lein run`.

Deployment
----------

Expand All @@ -146,7 +141,7 @@ Configuration

The configuration is loaded from `resources/config.edn`.

When running automated tests at the repl, or with `lein test`, a test environment
When running automated tests at the repl, or with `make test`, a test environment
is used to provide isolation. It can be found in `test/clojars/test/test_helper.clj`.

License
Expand Down
3 changes: 3 additions & 0 deletions bin/kaocha
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/usr/bin/env bash

clojure -M:defaults:dev:test "$@"
2 changes: 1 addition & 1 deletion bin/nvd-check
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,5 @@ clojure -Ttools install nvd-clojure/nvd-clojure '{:mvn/version "RELEASE"}' :as n

cd "$ROOT"
clojure -J-Dclojure.main.report=stderr -Tnvd nvd.task/check \
:classpath '"'"$(lein with-profile uberjar classpath)"'"' \
:classpath '"'"$(clojure -Spath -A:defaults)"'"' \
:config-filename '".nvd-clojure.edn"'
34 changes: 34 additions & 0 deletions build.clj
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
(ns build
(:require
[clojure.tools.build.api :as b])
(:import
(java.time
LocalDate)))

(def class-dir "target/classes")
(def basis (b/create-basis {:aliases [:default]}))
(def uber-file "target/clojars-web-standalone.jar")

(defn clean [_]
(b/delete {:path "target"}))

(defn uberjar [_]
(clean nil)
(b/copy-dir {:src-dirs ["src" "resources"]
:target-dir class-dir})
(println "Compiling...")
(b/compile-clj {:basis basis
:src-dirs ["src"]
:class-dir class-dir})
(printf "Building uberjar %s...\n" uber-file)
(b/uber {:class-dir class-dir
:uber-file uber-file
:basis basis
:main 'clojars.main}))

(def version (format "%s.%s" (LocalDate/now) (b/git-count-revs nil)))

(defn tag-release [_]
(printf "Tagging release version %s...\n" version)
(b/git-process {:git-args (format "tag %s" version)})
(b/git-process {:git-args (format "push origin %s" version)}))
112 changes: 112 additions & 0 deletions deps.edn
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
{:paths ["src" "resources"]
:mvn/repos
{"central" {:url "https://repo1.maven.org/maven2/"}
"clojars" {:url "https://repo.clojars.org/"}
;; Needed for com.github.kenglxn.qrgen/javase, which is a dependency of one-time.
;; See https://github.com/kenglxn/QRGen/issues/61
"jitpack" {:url "https://jitpack.io/"}}

:deps
{aero/aero {:mvn/version "1.1.6"}

buddy/buddy-core {:mvn/version "1.11.423"}

ch.qos.logback/logback-classic {:mvn/version "1.4.8"}
cheshire/cheshire {:mvn/version "5.10.1"}
clj-http/clj-http {:mvn/version "3.12.3"}
clj-stacktrace/clj-stacktrace {:mvn/version "0.2.8"}
clj-time/clj-time {:mvn/version "0.15.2"}
com.cemerick/friend {:mvn/version "0.2.3"
:exclusions [;; not used, excluded to address CVE-2007-1652, CVE-2007-1651
org.openid4java/openid4java-nodeps
;; not used, excluded to address CVE-2012-0881, CVE-2013-4002, CVE-2009-2625
net.sourceforge.nekohtml/nekohtml]}
com.cognitect.aws/api {:mvn/version "0.8.681"}
com.cognitect.aws/endpoints {:mvn/version "1.1.12.489"}
com.cognitect.aws/s3 {:mvn/version "847.2.1398.0"}
com.cognitect.aws/sqs {:mvn/version "847.2.1398.0"}
com.cognitect.aws/ssm {:mvn/version "847.2.1365.0"}
com.github.scribejava/scribejava-apis {:mvn/version "8.3.1"}
com.stuartsierra/component {:mvn/version "0.3.1"}

digest/digest {:mvn/version "1.4.10"}
duct/duct {:mvn/version "0.8.2"}
duct/hikaricp-component {:mvn/version "0.1.2"
:exclusions [org.slf4j/slf4j-nop]}

one-time/one-time {:mvn/version "0.7.0"
:exclusions [;; not needed on java 17, addresses CWE-120
com.github.jai-imageio/jai-imageio-core
;; not used, addresses CVE-2020-11987, CVE-2019-17566
org.apache.xmlgraphics/batik-dom
org.apache.xmlgraphics/batik-svggen]}
org.apache.commons/commons-email {:mvn/version "1.5"}
org.apache.lucene/lucene-core {:mvn/version "8.11.1"}
org.apache.lucene/lucene-analyzers-common {:mvn/version "8.11.1"}
org.apache.lucene/lucene-queryparser {:mvn/version "8.11.1"}
org.apache.maven/maven-model {:mvn/version "3.8.4"}
org.apache.maven/maven-repository-metadata {:mvn/version "3.8.4"}
org.clojure/clojure {:mvn/version "1.11.1"}
org.clojure/tools.logging {:mvn/version "1.2.4"}
org.clojure/tools.nrepl {:mvn/version "0.2.11"}
org.postgresql/postgresql {:mvn/version "42.6.0"}

net.cgrand/regex {:mvn/version "1.0.1"}

raven-clj/raven-clj {:mvn/version "1.6.0"}
ring/ring-core {:mvn/version "1.10.0"}
ring/ring-defaults {:mvn/version "0.3.3"}
ring/ring-jetty-adapter {:mvn/version "1.10.0"}
ring-jetty-component/ring-jetty-component {:mvn/version "0.3.1"}
ring-middleware-format/ring-middleware-format {:mvn/version "0.7.5"}

valip/valip {:mvn/version "0.2.0"}

yesql/yesql {:mvn/version "0.5.3"}}

:aliases {:defaults
;; We use override-deps to address CVEs
{:override-deps
{;; Addreses CVE-2022-42004, CVE-2022-42003, CVE-2021-46877, CVE-2020-36518
com.fasterxml.jackson.core/jackson-databind {:mvn/version "2.15.2"}
;; Addreses CVE-2019-10086, CVE-2014-0114
commons-beanutils/commons-beanutils {:mvn/version "1.9.4"}
;; Addresses CVE-2015-6420
commons-collections/commons-collections {:mvn/version "3.2.2"}

;; Addresses CVE-2015-0886
org.mindrot/jbcrypt {:mvn/version "0.4"}
;; Addresses CVE-2022-25857, CVE-2022-38749, CVE-2022-41854, CVE-2022-38751, CVE-2022-38752, CVE-2022-38750
org.yaml/snakeyaml {:mvn/version "1.33"}}}

:build {:deps {io.github.clojure/tools.build {:mvn/version "0.9.5"}}
:ns-default build}

:check {:extra-deps {athos/clj-check {:git/url "https://github.com/athos/clj-check.git"
:sha "0ca84df1357d71429243b99908303f45a934654c"}}
:main-opts ["-m" "clj-check.check"]}

:dev {:extra-deps
{clj-commons/pomegranate {:mvn/version "1.2.1"}

eftest/eftest {:mvn/version "0.5.9"}

kerodon/kerodon {:mvn/version "0.9.1"}

net.polyc0l0r/bote {:mvn/version "0.1.0"}
nubank/matcher-combinators {:mvn/version "3.8.6"}

org.clojure/tools.namespace {:mvn/version "1.2.0"}

reloaded.repl/reloaded.repl {:mvn/version "0.2.4"}

vvvvalvalval/scope-capture-nrepl {:mvn/version "0.3.1"}}
:extra-paths ["dev" "dev-resources" "test"]}

:migrate-db {:main-opts ["-m" "clojars.tools.migrate-db" "development"]}

:setup-dev-repo {:main-opts ["-m" "clojars.tools.setup-dev"]}

:test {:extra-deps
{lambdaisland/kaocha {:mvn/version "1.85.1342"}}
:main-opts ["-m" "kaocha.runner"]}}}
Loading