Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: fix circular dependencies related to assets.ts #1357

Open
wants to merge 33 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
c15ae88
fix types circular dependencies
schaeferka Oct 27, 2024
d829b52
saving wip - mutate rollback
schaeferka Oct 27, 2024
9fe9b3c
updated dependency toleration
schaeferka Oct 27, 2024
7573389
removed duplicate code
schaeferka Oct 28, 2024
eaada8f
updates to mutate
schaeferka Oct 28, 2024
7da425d
updated dependency tolerance
schaeferka Oct 28, 2024
60c303e
updated logging
schaeferka Oct 28, 2024
e3385f7
fixed assets and lib circular deps
schaeferka Oct 28, 2024
763c12f
fixed module-lib circular dep
schaeferka Oct 28, 2024
73f5281
assets circular deps fixes
schaeferka Oct 28, 2024
dc55f8b
Merge branch 'main' into 1340-assets-circular
schaeferka Oct 29, 2024
f1546a3
reconcile with main
schaeferka Oct 29, 2024
c74e641
code cleanup
schaeferka Oct 29, 2024
b33c3d8
cleaned up imports
schaeferka Oct 29, 2024
ac03330
merging in main
schaeferka Oct 30, 2024
85360c8
fixed imports
schaeferka Oct 30, 2024
3bf822d
added unit tests
schaeferka Oct 30, 2024
1f8fa1e
updated tests
schaeferka Oct 31, 2024
9ffe80d
updated unit tests
schaeferka Oct 31, 2024
fbbdb38
Merge branch 'main' into 1340-assets-circular
schaeferka Oct 31, 2024
bda0622
Merge branch 'main' into 1340-assets-circular
schaeferka Oct 31, 2024
3cd5186
Merge branch 'main' into 1340-assets-circular
schaeferka Oct 31, 2024
43f43bb
Merge branch 'main' into 1340-assets-circular
schaeferka Oct 31, 2024
d9f88e3
merge main
schaeferka Oct 31, 2024
881aefc
resolve merge conflicts
schaeferka Oct 31, 2024
feada7e
Merge branch 'main' into 1340-assets-circular
schaeferka Oct 31, 2024
ce76425
Update src/lib/assets/assetsDeployer.ts
schaeferka Oct 31, 2024
89c7b90
Update src/lib/assets/assetsDeployer.ts
schaeferka Oct 31, 2024
2ec753f
Update src/lib/assets/index.test.ts
schaeferka Oct 31, 2024
8bbdc39
format fix
schaeferka Oct 31, 2024
8e2d951
fixed customImage
schaeferka Nov 1, 2024
a434285
Merge branch 'main' into 1340-assets-circular
schaeferka Nov 1, 2024
ba235a1
Merge branch 'main' into 1340-assets-circular
samayer12 Nov 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .github/workflows/dependency-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,13 @@ jobs:
- name: Circular Dependency Check
run: |
npx madge --circular --ts-config tsconfig.json --extensions ts,js src/ > tmp.log || true # Force exit 0 for post-processing
tail -n +4 tmp.log > circular-deps.log
if [ $(wc -l < circular-deps.log) -gt 11 ]; then
echo "circular-deps.log has more than 11 circular dependencies."
tail -n +6 tmp.log > circular-deps.log
if [ $(wc -l < circular-deps.log) -gt 6 ]; then
echo "circular-deps.log has more than 6 circular dependencies."
wc -l circular-deps.log
exit 1
else
echo "circular-deps.log has 11 or fewer circular dependencies."
echo "circular-deps.log has 6 or fewer circular dependencies."

exit 0
fi
11 changes: 7 additions & 4 deletions src/cli/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
import { promises as fs } from "fs";
import { basename, dirname, extname, resolve } from "path";
import { createDockerfile } from "../lib/included-files";
import { Assets } from "../lib/assets";
import { AssetsConfig } from "../lib/assets/assetsConfig";
import { AssetsDeployer } from "../lib/assets/assetsDeployer";
import { dependencies, version } from "./init/templates";
import { RootCmd } from "./root";
import { peprFormat } from "./format";
Expand Down Expand Up @@ -70,7 +71,7 @@
["admin", "scoped"],
),
)
.action(async opts => {

Check warning on line 74 in src/cli/build.ts

View workflow job for this annotation

GitHub Actions / format

Async arrow function has too many statements (53). Maximum allowed is 20

Check warning on line 74 in src/cli/build.ts

View workflow job for this annotation

GitHub Actions / format

Async arrow function has a complexity of 15. Maximum allowed is 10
// assign custom output directory if provided
if (opts.outputDir) {
outputDir = opts.outputDir;
Expand Down Expand Up @@ -128,7 +129,7 @@
}

// Generate a secret for the module
const assets = new Assets(
const assetsConfig = new AssetsConfig(
{
...cfg.pepr,
appVersion: cfg.version,
Expand All @@ -139,6 +140,8 @@
path,
);

const assets = new AssetsDeployer(assetsConfig);

// If registry is set to Iron Bank, use Iron Bank image
if (opts?.registry === "Iron Bank") {
console.info(
Expand All @@ -149,7 +152,7 @@

// if image is a custom image, use that instead of the default
if (image !== "") {
assets.image = image;
assetsConfig.image = image;
}

// Ensure imagePullSecret is valid
Expand All @@ -170,7 +173,7 @@

try {
// wait for capabilities to be loaded and test names
validateCapabilityNames(assets.capabilities);
validateCapabilityNames(assetsConfig.capabilities);
} catch (e) {
console.error(`Error loading capability:`, e);
process.exit(1);
Expand Down Expand Up @@ -243,7 +246,7 @@
};
}

export async function buildModule(reloader?: Reloader, entryPoint = peprTS, embed = true) {

Check warning on line 249 in src/cli/build.ts

View workflow job for this annotation

GitHub Actions / format

Async function 'buildModule' has too many statements (36). Maximum allowed is 20
try {
const { cfg, modulePath, path, uuid } = await loadModule(entryPoint);

Expand Down Expand Up @@ -349,7 +352,7 @@
const conflicts = [...out.matchAll(pgkErrMatch)];

// If the regex didn't match, leave a generic error
if (conflicts.length < 1) {

Check warning on line 355 in src/cli/build.ts

View workflow job for this annotation

GitHub Actions / format

Blocks are nested too deeply (4). Maximum allowed is 3
console.info(
`\n\tOne or more imported Pepr Capabilities seem to be using an incompatible version of Pepr.\n\tTry updating your Pepr Capabilities to their latest versions.`,
"Version Conflict",
Expand Down
15 changes: 8 additions & 7 deletions src/cli/deploy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@

import prompt from "prompts";

import { Assets } from "../lib/assets";
import { AssetsConfig } from "../lib/assets/assetsConfig";
import { AssetsDeployer } from "../lib/assets/assetsDeployer";
import { buildModule } from "./build";
import { RootCmd } from "./root";
import { validateCapabilityNames, namespaceDeploymentsReady } from "../lib/helpers";
Expand All @@ -23,7 +24,7 @@
.option("--docker-email <email>", "Email for Docker registry")
.option("--docker-password <password>", "Password for Docker registry")
.option("--force", "Force deploy the module, override manager field")
.action(async opts => {

Check warning on line 27 in src/cli/deploy.ts

View workflow job for this annotation

GitHub Actions / format

Async arrow function has too many statements (26). Maximum allowed is 20

Check warning on line 27 in src/cli/deploy.ts

View workflow job for this annotation

GitHub Actions / format

Async arrow function has a complexity of 15. Maximum allowed is 10
let imagePullSecret: ImagePullSecret | undefined;

if (
Expand Down Expand Up @@ -73,27 +74,27 @@

// Build the module
const { cfg, path } = await buildModule();

// Generate a secret for the module
const webhook = new Assets(
// Initialize AssetsConfig and AssetsDeployer
const assetsConfig = new AssetsConfig(
{
...cfg.pepr,
description: cfg.description,
},
path,
);
const assetsDeployer = new AssetsDeployer(assetsConfig);

if (opts.image) {
webhook.image = opts.image;
assetsConfig.image = opts.image;
}

// Identify conf'd webhookTimeout to give to deploy call
const timeout = cfg.pepr.webhookTimeout ? cfg.pepr.webhookTimeout : 10;

try {
await webhook.deploy(opts.force, timeout);
await assetsDeployer.deploy(opts.force, timeout);
// wait for capabilities to be loaded and test names
validateCapabilityNames(webhook.capabilities);
validateCapabilityNames(assetsConfig.capabilities);
// Wait for the pepr-system resources to be fully up
await namespaceDeploymentsReady();
console.info(`✅ Module deployed successfully`);
Expand Down
20 changes: 12 additions & 8 deletions src/cli/dev.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ import { ChildProcess, fork } from "child_process";
import { promises as fs } from "fs";
import prompt from "prompts";
import { validateCapabilityNames } from "../lib/helpers";
import { Assets } from "../lib/assets";
import { AssetsConfig } from "../lib/assets/assetsConfig";
import { AssetsDeployer } from "../lib/assets/assetsDeployer";
import { buildModule, loadModule } from "./build";
import { RootCmd } from "./root";
import { K8s, kind } from "kubernetes-fluent-client";
Expand Down Expand Up @@ -34,8 +35,8 @@ export default function (program: RootCmd) {
// Build the module
const { cfg, path } = await loadModule();

// Generate a secret for the module
const webhook = new Assets(
// Initialize AssetsConfig with the configuration and path
const assetsConfig = new AssetsConfig(
{
...cfg.pepr,
description: cfg.description,
Expand All @@ -44,9 +45,12 @@ export default function (program: RootCmd) {
opts.host,
);

// Create an instance of AssetsDeployer
const assetsDeployer = new AssetsDeployer(assetsConfig);

// Write the TLS cert and key to disk
await fs.writeFile("insecure-tls.crt", webhook.tls.pem.crt);
await fs.writeFile("insecure-tls.key", webhook.tls.pem.key);
await fs.writeFile("insecure-tls.crt", assetsConfig.tls.pem.crt);
await fs.writeFile("insecure-tls.key", assetsConfig.tls.pem.key);

try {
let program: ChildProcess;
Expand All @@ -59,11 +63,11 @@ export default function (program: RootCmd) {
console.info(`Running module ${path}`);

// Deploy the webhook with a 30 second timeout for debugging, don't force
await webhook.deploy(false, 30);
await assetsDeployer.deploy(false, 30);

try {
// wait for capabilities to be loaded and test names
validateCapabilityNames(webhook.capabilities);
validateCapabilityNames(assetsConfig.capabilities);
} catch (e) {
console.error(`Error validating capability names:`, e);
process.exit(1);
Expand All @@ -74,7 +78,7 @@ export default function (program: RootCmd) {
...process.env,
LOG_LEVEL: "debug",
PEPR_MODE: "dev",
PEPR_API_TOKEN: webhook.apiToken,
PEPR_API_TOKEN: assetsConfig.apiToken,
PEPR_PRETTY_LOGS: "true",
SSL_KEY_PATH: "insecure-tls.key",
SSL_CERT_PATH: "insecure-tls.crt",
Expand Down
99 changes: 99 additions & 0 deletions src/lib/assets/assetsConfig.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
// SPDX-License-Identifier: Apache-2.0
// SPDX-License-IdentifierText: 2023-Present The Pepr Authors

import crypto from "crypto";
import { genTLS, TLSOut } from "../tls";
import { ModuleConfig } from "../module";
import { CapabilityExport } from "../types";
import { WebhookIgnore } from "../k8s";
import { AssetsConfig } from "./assetsConfig";
import { jest, describe, beforeEach, it, expect } from "@jest/globals";

jest.mock("crypto");
jest.mock("../tls");

describe("AssetsConfig", () => {
const mockUUID = "test-uuid";
const mockAlwaysIgnore: WebhookIgnore = {};
const mockPeprVersion = "1.0.0";
const mockPath = "/path/to/module";
const mockHost = "example.com";
const mockConfig: ModuleConfig = {
uuid: mockUUID,
alwaysIgnore: mockAlwaysIgnore,
peprVersion: mockPeprVersion,
};

const mockTLS: TLSOut = {
crt: "mockCert",
key: "mockKey",
ca: "",
pem: {
ca: "",
crt: "",
key: "",
},
};

beforeEach(() => {
jest.resetAllMocks();
(genTLS as jest.Mock).mockReturnValue(mockTLS);
(crypto.randomBytes as jest.Mock).mockReturnValue({ toString: () => "mockApiToken" });
});

describe("Initialization", () => {
it("should initialize with correct values", () => {
const assetsConfig = new AssetsConfig(mockConfig, mockPath, mockHost);

expect(assetsConfig.name).toBe(`pepr-${mockUUID}`);
expect(assetsConfig.buildTimestamp).toMatch(/^\d+$/);
expect(assetsConfig.alwaysIgnore).toBe(mockAlwaysIgnore);
expect(assetsConfig.image).toBe(`ghcr.io/defenseunicorns/pepr/controller:v${mockPeprVersion}`);
expect(assetsConfig.tls).toEqual(mockTLS);
expect(assetsConfig.apiToken).toBe("mockApiToken");
});

it("should handle undefined host by defaulting to cluster internal URL", () => {
const assetsConfig = new AssetsConfig(mockConfig, mockPath);

expect(assetsConfig.tls).toEqual(mockTLS);
expect(genTLS).toHaveBeenCalledWith("pepr-test-uuid.pepr-system.svc");
});
});

describe("Setting Properties", () => {
it("should set hash correctly", () => {
const assetsConfig = new AssetsConfig(mockConfig, mockPath);
const testHash = "test-hash";

assetsConfig.setHash(testHash);

expect(assetsConfig.hash).toBe(testHash);
});

it("should allow setting capabilities", () => {
const assetsConfig = new AssetsConfig(mockConfig, mockPath);
const mockCapabilities: CapabilityExport[] = [
{
name: "test",
description: "test",
namespaces: [],
bindings: [],
hasSchedule: false,
},
];

assetsConfig.capabilities = mockCapabilities;

expect(assetsConfig.capabilities).toBe(mockCapabilities);
});
});

describe("Capabilities", () => {
it("should have capabilities as undefined initially", () => {
const assetsConfig = new AssetsConfig(mockConfig, mockPath);

expect(assetsConfig.capabilities).toBeUndefined();
});
});
});
38 changes: 38 additions & 0 deletions src/lib/assets/assetsConfig.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: 2023-Present The Pepr Authors

import crypto from "crypto";
import { TLSOut, genTLS } from "../tls";
import { ModuleConfig } from "../module";
import { CapabilityExport } from "../types";
import { WebhookIgnore } from "../k8s";

export class AssetsConfig {
readonly name: string;
readonly tls: TLSOut;
readonly apiToken: string;
readonly alwaysIgnore!: WebhookIgnore;
capabilities!: CapabilityExport[];

image: string;
buildTimestamp: string;
hash: string;

constructor(
readonly config: ModuleConfig,
readonly path: string,
readonly host?: string,
) {
this.name = `pepr-${config.uuid}`;
this.buildTimestamp = `${Date.now()}`;
this.alwaysIgnore = config.alwaysIgnore;
this.image = `ghcr.io/defenseunicorns/pepr/controller:v${config.peprVersion}`;
this.hash = "";
this.tls = genTLS(this.host || `${this.name}.pepr-system.svc`);
this.apiToken = crypto.randomBytes(32).toString("hex");
}

setHash = (hash: string) => {
this.hash = hash;
};
}
Loading
Loading