Skip to content

Commit

Permalink
adding the checkers
Browse files Browse the repository at this point in the history
  • Loading branch information
perryqh committed Aug 29, 2024
1 parent d55cce0 commit ef87ab5
Show file tree
Hide file tree
Showing 4 changed files with 253 additions and 85 deletions.
228 changes: 228 additions & 0 deletions CHECKERS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,228 @@
# Checkers
## Privacy Checker
The privacy checker extension was originally extracted from [packwerk](https://github.com/Shopify/packwerk).

A package's privacy boundary is violated when there is a reference to the package's private constants from a source outside the package.

To enforce privacy for your package, set `enforce_privacy` to `true` or `strict` on your pack:

```yaml
# components/merchandising/package.yml
enforce_privacy: true
```
Setting `enforce_privacy` to `true` will make all references to private constants in your package a violation.

Setting `enforce_privacy` to `strict` will forbid all references to private constants in your package. **This includes violations that have been added to other packages' `package_todo.yml` files.**

Note: You will need to remove all existing privacy violations before setting `enforce_privacy` to `strict`.

### Using public folders
You may enforce privacy either way mentioned above and still expose a public API for your package by placing constants in the public folder, which by default is `app/public`. The constants in the public folder will be made available for use by the rest of the application.

### Defining your own public folder

You may prefer to override the default public folder, you can do so on a per-package basis by defining a `public_path`.

Example:

```yaml
public_path: my/custom/path/
```

### Defining public constants through sigil

> [!WARNING]
> This way of of defining the public API of a package should be considered WIP. It is not supported by all tooling in the RubyAtScale ecosystem, as @alexevanczuk pointed out in a [comment on the PR](https://github.com/rubyatscale/packwerk-extensions/pull/35#discussion_r1334331797):
>
> There are a couple of other places that will require changes related to this sigil. Namely, everything that is coupled to the public folder implementation of privacy.
>
> In the rubyatscale org:
>
> * pack_stats, example https://github.com/rubyatscale/pack_stats/blob/main/lib/pack_stats/private/metrics/public_usage.rb. (IMO though we can just remove this metric – it has never been useful)
> * Other places that mention public_path or app/public.
> * Org wide search for app/public link
> * Org wide search for public_path link
> * packs (the Rust port of packwerk – I could take this one over unless someone is interested in implementing whatever we come up with there



You may make individual files public within a private package by usage of a comment within the first 5 lines of the `.rb` file containing `pack_public: true`.

Example:

```ruby
# pack_public: true
module Foo
class Update
end
end
```
Now `Foo::Update` is considered public even though the `foo` package might be set to `enforce_privacy: (true || strict)`.

It's important to note that when combining `public_api: true` with the declaration of `private_constants`,
`packwerk validate` will raise an exception if both are used for the same constant. This must be resolved by removing
the sigil from the `.rb` file or removing the constant from the list of `private_constants`.

If you are using rubocop, it may be configured in such a way that there must be an empty line after the magic keywords at the top of the file. Currently, this extension is not modifying rubocop in any way so it does not recognize `pack_public: true` as a valid magic keyword option. That means placing it at the end of the magic keywords will throw a rubocop exception. However, you can place it first in the list to avoid an exception in rubocop.
```
-----
# typed: ignore
# frozen_string_literal: true
# pack_public: true
class Foo
...
end => Layout/EmptyLineAfterMagicComment: Add an empty line after magic comments.
------
# typed: ignore
# frozen_string_literal: true
# pack_public: true
class Foo
...
end => Less than ideal. This won't raise an issue in rubocop, however, only the first 5 lines are scanned for the magic comment of pack_public so there is risk at it being missed. It also is requiring extra empty lines in the group of magic comments.
-----
# pack_public: true
# typed: ignore
# frozen_string_literal: true
class Foo
...
end => Ideal solution. No exceptions from rubocop and very low risk of the magic comment being out of range since
```

### Using specific private constants
Sometimes it is desirable to only enforce privacy on a subset of constants in a package. You can do so by defining a `private_constants` list in your package.yml. Note that `enforce_privacy` must be set to `true` or `'strict'` for this to work.

### Ignore strict mode for violation coming from specific path patterns
If you want to activate `'strict'` mode on your package but have a few privacy violations you know you will deal with later,
you can set a list of patterns to exclude.

```yaml
enforce_privacy: strict
strict_privacy_ignored_patterns:
- engines/another_engine/test/**/*
```

In this example, violations on constants of your engine referenced in those files `engines/another_engine/test/**/*` will not fail Packwerk checks.

### Package Privacy violation
Packwerk thinks something is a privacy violation if you're referencing a constant, class, or module defined in the private implementation (i.e. not the public folder) of another package. We care about these because we want to make sure we only use parts of a package that have been exposed as public API.

#### Interpreting Privacy violation

> /Users/JaneDoe/src/github.com/sample-project/user/app/controllers/labels_controller.rb:170:30
> Privacy violation: '::Billing::CarrierInvoiceTransaction' is private to 'billing' but referenced from 'user'.
> Is there a public entrypoint in 'billing/app/public/' that you can use instead?
>
> Inference details: 'Billing::CarrierInvoiceTransaction' refers to ::Billing::CarrierInvoiceTransaction which seems to be defined in billing/app/models/billing/carrier_invoice_transaction.rb.

There has been a privacy violation of the package `billing` in the package `user`, through the use of the constant `Billing::CarrierInvoiceTransaction` in the file `user/app/controllers/labels_controller.rb`.

#### Suggestions
You may be accessing the implementation of a piece of functionality that is supposed to be accessed through a public interface on the package. Try to use the public interface instead. A package’s public interface should be defined in its `app/public` folder and documented.

The functionality you’re looking for may not be intended to be reused across packages at all. If there is no public interface for it but you have a good reason to use it from outside of its package, find the people responsible for the package and discuss a solution with them.

## Visibility Checker
The visibility checker can be used to allow a package to be a private implementation detail of other packages.

To enforce visibility for your package, set `enforce_visibility` to `true` on your pack and specify `visible_to` for other packages that can use your package.

```yaml
# components/merchandising/package.yml
enforce_visibility: true
visible_to:
- components/other_package
```

## Folder-Privacy Checker
The folder privacy checker can be used to allow a package to be private to their sibling packs and parent packs and will create todos if used by any other package.

To enforce folder privacy for your package, set `enforce_folder_privacy` to `true` on your pack.

```yaml
# components/merchandising/package.yml
enforce_folder_privacy: true
```

Here is an example of paths and whether their use of `packs/b/packs/e` is OK or not, assuming that protects itself via `enforce_folder_privacy`

```
. OK (parent of parent)
packs/a VIOLATION
packs/b OK (parent)
packs/b/packs/d OK (sibling)
packs/b/packs/e ENFORCE_NESTED_VISIBILITY: TRUE
packs/b/packs/e/packs/f VIOLATION
packs/b/packs/e/packs/g VIOLATION
packs/b/packs/h OK (sibling)
packs/c VIOLATION
```
## Layer Checker
The layer checker can be used to enforce constraints on what can depend on what.
To enforce layers for your package, first define the `layers` in `packwerk.yml`, for example:
```
layers:
- package
- utility
```
Then, turn on the checker in your package:
```yaml
# components/merchandising/package.yml
enforce_layers: true
layer: utility
```

Now this pack can only depend on other utility packages.

# Fixing `pks check` privacy and dependency violations
[Violation Flow Chart](https://drive.google.com/file/d/1Y1x0ncF6EsJxj9fM2wm-k35auXaxRjEB/view)

# Enforcement Globs Ignore
`enforcement_globs_ignore` can be used to specify gitignore-style rules for not enforcing violations.

### Examples

```yml
# packs/product_services/serv1/foo/package.yml
enforce_privacy: true
enforce_visibility: true

enforcement_globs_ignore:
- enforcements:
- privacy
- visiblity
ignores:
- "**/*"
# Enforce incoming privacy and visibility violation references _only_ in `pks/product_services/serv1/**/*`
- "!packs/product_services/serv1/**/*"
reason: "It was decided only to fix incoming violations from serv1. See ticket #232"
```
```yml
# packs/pack2/package.yml
enforce_dependencies: true
dependencies:
# not required because of the below enforcement_globs_ignore
# - packs/pack1
# required because of the enforcement_globs_ignore exception line
- packs/pack3

enforcement_globs_ignore:
- enforcements:
- dependency
ignores:
- "**/*"
# Enforce outgoing dependency violation references _only_ to `pks/pack3/**/*`
- "!packs/pack3/**/*"
reason: "The other dependency violations are fine as those packs will be absorbed into this one."
```
25 changes: 6 additions & 19 deletions INSTALLATION.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,12 @@
# Installation
## Option 1:
- Install [dotslash](https://dotslash-cli.com/docs/installation/)
- Download the latest packs release dotslash `pks` file. Example: https://github.com/rubyatscale/pks/releases/tag/v0.2.21/pks
- Save the `pks` file to your ruby project's bin/ directory. You'll then have a `bin/pks` file in your project.
- Use `bin/pks` to execute the CLI.

## Option 2:
- Install Rust: https://www.rust-lang.org/tools/install
- Note: If `which cargo` returns a path, skip this step!
- TLDR: `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`, and you're done!
- `cargo install pks` (it's like `gem install`)

## Option 2:
(Mac only – for other platforms, please create an issue/PR or try option 1.)

- Go to https://github.com/alexevanczuk/packs/releases
- Download the `packs` asset and run `chmod +x path/to/packs`. This makes the asset executable on your machine.
- Open the containing directory, right click on the binary, click open, and then accept the warning message that says its from an unknown developer (it's me!)
- Execute `path/to/packs` to see the CLI help message.

You can add `path/to/packs` to your `PATH` so it's available in every terminal session.

## Option 3 (coming soon):
I'm looking into installing via `brew` or as a native ruby gem extension. More coming soon!

## Option 4:
- Install [dotslash](https://dotslash-cli.com/docs/installation/)
- Download the latest packs release dotslash `pks` file. Example: https://github.com/alexevanczuk/packs/releases/download/v0.2.8/pks
- Save the `pks` file to your ruby project's bin/ directory. You'll then have a `bin/pks` file in your project.
- Use `bin/pks` to execute the CLI.
81 changes: 17 additions & 64 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,21 @@
[![CI](https://github.com/rubyatscale/pks/actions/workflows/ci.yml/badge.svg)](https://github.com/rubyatscale/pks/actions)
[![Security Audit](https://github.com/rubyatscale/pks/actions/workflows/audit.yml/badge.svg)](https://github.com/rubyatscale/pks/actions?query=workflow%3A%22Security+audit%22++)

A 100% Rust implementation of [packwerk](https://github.com/Shopify/packwerk), a gradual modularization platform for Ruby.
A 100% Rust implementation of [packwerk](https://github.com/Shopify/packwerk) and [packwerk-extensions](https://github.com/rubyatscale/packwerk-extensions), a gradual modularization platform for Ruby.

Currently, it ships the following checkers to help improve the boundaries between packages. These checkers are:
- A `dependency` checker requires that a pack specifies packs on which it depends. Cyclic dependencies are not allowed. See [packwerk](https://github.com/Shopify/packwerk)
- A `privacy` checker that ensures other packages are using your package's public API
- A `visibility` checker that allows packages to be private except to an explicit group of other packages.
- A `folder_privacy` checker that allows packages to their sibling packs and parent pack (to be used in an application that uses folder packs)
- A `layer` (formerly `architecture`) checker that allows packages to specify their "layer" and requires that each layer only communicate with layers below it.

See [Checkers](CHECKERS.md) for detailed descriptions.

# Fork
This repo was forked directly from https://github.com/alexevanczuk/packs

# Goals:
## Serve as a drop-in replacement for `packwerk` on most projects
- Currently can serve as a drop-in replacement on Gusto's extra-large Rails monolith
- This is a work in progress! Please see [Verification](#verification) for instructions on how to verify the output of `pks` is the same as `packwerk`.

## Run 20x faster than `packwerk` on most projects
- Currently ~10-20x as fast as the ruby implementation. See [BENCHMARKS.md](https://github.com/rubyatscale/pks/blob/main/BENCHMARKS.md).
Expand Down Expand Up @@ -40,7 +46,7 @@ Commands:
update Update package_todo.yml files with the current violations
validate Look for validation errors in the codebase
add-dependency Add a dependency from one pack to another
check-unnecessary-dependencies Check for dependencies that when removed produce no violations.
check-unused-dependencies Check for dependencies that when removed produce no violations.
lint-package-yml-files Lint package.yml files
expose-monkey-patches Expose monkey patches of the Ruby stdlib, gems your app uses, and your application itself
delete-cache `rm -rf` on your cache directory, default `tmp/cache/packwerk`
Expand All @@ -59,6 +65,11 @@ Options:
-V, --version Print version
```


# Fixing `pks check` privacy and dependency violations
[Violation Flow Chart](https://drive.google.com/file/d/1Y1x0ncF6EsJxj9fM2wm-k35auXaxRjEB/view)


# Installation
See [INSTALLATION.md](https://github.com/rubyatscale/pks/blob/main/INSTALLATION.md)

Expand All @@ -73,25 +84,6 @@ Directions:
- Follow [INSTALLATION.md](https://github.com/rubyatscale/pks/blob/main/INSTALLATION.md) instructions to install `pks`
- Follow the [configuration](https://github.com/rubyatscale/packwerk-vscode/tree/main#configuration) directions to configure the extension to use `pks` instead of the ruby gem by setting the executable to `pks check`

# Verification
As `pks` is still a work-in-progress, it's possible it will not produce the same results as the ruby implementation (see [Not Yet Supported](#not-yet-supported)). If so, please file an issue – I'd love to try to support your use case!

Instructions:
- Follow the directions above to install `pks`
- Run `pks update`
- Confirm the output of `git diff` is empty
- Please file an issue if it's not!


# New to Rust?
Me too! This is my first Rust project, so I'd love to have feedback, advice, and contributions!

Rust is a low-level language with high-level abstractions, a rich type system, with a focus on memory safety through innovative compile-time checks on memory usage.

If you're new to Rust, don't be intimidated! [https://www.rust-lang.org](https://www.rust-lang.org/learn) has tons of great learning resources.

If you'd like to contribute but don't know where to start, please reach out! I'd love to help you get started.

# Not yet supported
- custom inflections
- custom load paths
Expand All @@ -116,46 +108,6 @@ autoload_roots:
packs/foo/app/domain: "::Foo"
```
## Enforcement Globs Ignore
`enforcement_globs_ignore` can be used to specify gitignore-style rules for not enforcing violations.

### Examples

```yml
# packs/product_services/serv1/foo/package.yml
enforce_privacy: true
enforce_visibility: true
enforcement_globs_ignore:
- enforcements:
- privacy
- visiblity
ignores:
- "**/*"
# Enforce incoming privacy and visibility violation references _only_ in `pks/product_services/serv1/**/*`
- "!packs/product_services/serv1/**/*"
reason: "It was decided only to fix incoming violations from serv1. See ticket #232"
```
```yml
# packs/pack2/package.yml
enforce_dependencies: true
dependencies:
# not required because of the below enforcement_globs_ignore
# - packs/pack1
# required because of the enforcement_globs_ignore exception line
- packs/pack3

enforcement_globs_ignore:
- enforcements:
- dependency
ignores:
- "**/*"
# Enforce outgoing dependency violation references _only_ to `pks/pack3/**/*`
- "!packs/pack3/**/*"
reason: "The other dependency violations are fine as those packs will be absorbed into this one."
```
## "check" error messages
The error messages resulting from running `pks check` can be customized with mustache-style interpolation. The available
variables are:
Expand Down Expand Up @@ -186,6 +138,7 @@ checker_overrides:
See [BENCHMARKS.md](https://github.com/rubyatscale/pks/blob/main/BENCHMARKS.md)
# Kudos
- @alexevanczuk for https://github.com/alexevanczuk/packs
- Current (@gmcgibbon, @rafaelfranca), and Ex-Shopifolks (@exterm, @wildmaples) for open-sourcing and maintaining `packwerk`
- Gusties, and the [Ruby/Rails Modularity Slack Server](https://join.slack.com/t/rubymod/shared_invite/zt-1dgyrxji9-sihGNX43mVh5T6tw18hFaQ), for continued feedback and support
- @mzruya for the initial implementation and Rust inspiration
4 changes: 2 additions & 2 deletions dev/notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
- Privacy violation inversion?

## Performance
Although `packs` is intended to be fast, there are ways it can be made a lot faster!
- Conditional cache usage. For example, implemented as an LSP, packs could always use cache and only bust specific caches (asychronously) when certain events (e.g. file changes) are received.
Although `pks` is intended to be fast, there are ways it can be made a lot faster!
- Conditional cache usage. For example, implemented as an LSP, pks could always use cache and only bust specific caches (asychronously) when certain events (e.g. file changes) are received.
- By using modified time, we can avoid opening the entire file and parsing it and calculating the md5 hash. It's possible this would not be a meaningful performance improvement.

### Improved use of references (less cloning)
Expand Down

0 comments on commit ef87ab5

Please sign in to comment.