Skip to content

Commit

Permalink
Limit creation of new projects to administrators, a la gypsum. (#10)
Browse files Browse the repository at this point in the history
This aligns with the gypsum philosophy and gives administrators more control by
preventing users from just creating loads of projects; any projects that are
created must have some owners that are held accountable for usage. The change
also simplifies matters by separating the upload from the setup of permissions.

We also eliminated the idea of incremental projects. Given that an
administrator has to create the project anyway, there isn't much benefit to
automating the name creation. While we're here, we might as well get rid of the
incremental versions and simplify the code.
  • Loading branch information
LTLA authored Apr 9, 2024
1 parent 2c3246c commit cf896e8
Show file tree
Hide file tree
Showing 6 changed files with 514 additions and 754 deletions.
39 changes: 22 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,9 +151,24 @@ Specifically, users should write the JSON request body to a file inside the stag
Once the write is complete, this file can be renamed to a file with said prefix.
This ensures that the Gobbler does not read a partially-written file.

### Creating projects (admin)

Administrators are responsible for creating new projects within the registry.
This is done using the write-and-rename paradigm to create a file with the `request-create_project-` prefix.
This file should be JSON-formatted with the following properties:

- `project`: string containing the name of the new project.
This should not contain `/`, `\`, or `.`.
- `permissions` (optional): an object containing either or both of `owners` and `uploaders`.
Each of these properties has the same type as described [above](#permissions).
If `owners` is not supplied, it is automatically set to a length-1 array containing only the uploading user.
This property is ignored when uploading a new version of an asset to an existing project.

On success, a new project is created with the designated permissions and a JSON formatted file will be created in `responses` with the `status` property set to `SUCCESS`.

### Uploads and updates

To upload a new version of an asset of a project (either in an existing project, or in a new project), users should create a temporary directory within the staging directory.
To upload a new version of an asset of a project, users should create a temporary directory within the staging directory.
The directory may have any name but should avoid starting with `request-`.
Files within this temporary directory will be transferred to the appropriate subdirectory within the registry, subject to the following rules:

Expand All @@ -164,26 +179,16 @@ Files within this temporary directory will be transferred to the appropriate sub
Once this directory is constructed and populated, the user should use the write-and-rename paradigm to create a file with the `request-upload-` prefix.
This file should be JSON-formatted with the following properties:

- `project`: string containing the explicit name of the project.
The string should start with a lower-case letter to distinguish from prefixed names constructed by incrementing series.
This may also be missing, in which case `prefix` should be supplied.
- `prefix` (optional): string containing an all-uppercase project prefix,
used to derive a project name from an incrementing series if `project` is not supplied.
Ignored if `project` is provided.
- `project`: string containing the name of an existing project.
- `asset`: string containing the name of the asset.
- `version` (optional): string containing the name of the version.
This may be missing, in which case the version is named according to an incrementing series for that project-asset combination.
This should not contain `/`, `\`, or `.`.
- `version`: string containing the name of the version.
This should not contain `/`, `\`, or `.`.
- `source`: string containing the name of the temporary directory, itself containing the files to be uploaded for this version of the asset.
This temporary directory is expected to be inside the staging directory.
- `permissions` (optional): an object containing either or both of `owners` and `uploaders`.
Each of these properties has the same type as described [above](#permissions).
If `owners` is not supplied, it is automatically set to a length-1 array containing only the uploading user.
This property is ignored when uploading a new version of an asset to an existing project.
- `on_probation` (optional): boolean specifying whether this version of the asset should be considered as probational.

On success, the files will be transferred to the registry and a JSON formatted file will be created in `responses`.
This will have the `status` property set to `SUCCESS` and the `project` and `version` strings set to the names of the project and version, respectively.
These will be the same as the request parameters if supplied, otherwise they are created from the relevant incrementing series.
On success, the files will be transferred to the registry and a JSON formatted file will be created in `responses` with the `status` property set to `SUCCESS`.

### Setting permissions

Expand All @@ -195,7 +200,7 @@ This file should be JSON-formatted with the following properties:
Each of these properties has the same type as described [above](#permissions).
If any property is missing, the value in the existing permissions is used.

On success, the permissions in the registry are modified and a JSON formatted file will be created in `responses` with the `status` property set to `SUCCESS`..
On success, the permissions in the registry are modified and a JSON formatted file will be created in `responses` with the `status` property set to `SUCCESS`.

### Handling probation

Expand Down
100 changes: 100 additions & 0 deletions create.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
package main

import (
"fmt"
"time"
"path/filepath"
"os"
"encoding/json"
"strconv"
"errors"
)

func createProjectHandler(reqpath string, globals *globalConfiguration) error {
req_user, err := identifyUser(reqpath)
if err != nil {
return fmt.Errorf("failed to find owner of %q; %w", reqpath, err)
}
if !isAuthorizedToAdmin(req_user, globals.Administrators) {
return fmt.Errorf("user %q is not authorized to create a project", req_user)
}

request := struct {
Project *string `json:"project"`
Permissions *unsafePermissionsMetadata `json:"permissions"`
}{}

// Reading in the request.
handle, err := os.ReadFile(reqpath)
if err != nil {
return fmt.Errorf("failed to read %q; %w", reqpath, err)
}
err = json.Unmarshal(handle, &request)
if err != nil {
return fmt.Errorf("failed to parse JSON from %q; %w", reqpath, err)
}

if request.Project == nil {
return fmt.Errorf("expected a 'project' property in %q", reqpath)
}
project := *(request.Project)

return createProject(project, request.Permissions, req_user, globals)
}

func createProject(project string, inperms *unsafePermissionsMetadata, req_user string, globals *globalConfiguration) error {
err := isBadName(project)
if err != nil {
return fmt.Errorf("invalid project name; %w", err)
}

// Creating a new project from a pre-supplied name.
project_dir := filepath.Join(globals.Registry, project)
if _, err = os.Stat(project_dir); !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("project %q already exists", project)
}

// No need to lock before MkdirAll, it just no-ops if the directory already exists.
err = os.MkdirAll(project_dir, 0755)

globals.Locks.LockPath(project_dir, 1000 * time.Second)
if err != nil {
return fmt.Errorf("failed to acquire the lock on %q; %w", project_dir, err)
}
defer globals.Locks.UnlockPath(project_dir)

perms := permissionsMetadata{}
if inperms != nil && inperms.Owners != nil {
perms.Owners = inperms.Owners
} else {
perms.Owners = []string{ req_user }
}
if inperms != nil && inperms.Uploaders != nil {
san, err := sanitizeUploaders(inperms.Uploaders)
if err != nil {
return fmt.Errorf("invalid 'permissions.uploaders' in the request details; %w", err)
}
perms.Uploaders = san
} else {
perms.Uploaders = []uploaderEntry{}
}

err = dumpJson(filepath.Join(project_dir, permissionsFileName), &perms)
if err != nil {
return fmt.Errorf("failed to write permissions for %q; %w", project_dir, err)
}

// Dumping a mock quota and usage file for consistency with gypsum.
// Note that the quota isn't actually enforced yet.
err = os.WriteFile(filepath.Join(project_dir, "..quota"), []byte("{ \"baseline\": 1000000000, \"growth_rate\": 1000000000, \"year\": " + strconv.Itoa(time.Now().Year()) + " }"), 0755)
if err != nil {
return fmt.Errorf("failed to write quota for '" + project_dir + "'; %w", err)
}

err = os.WriteFile(filepath.Join(project_dir, usageFileName), []byte("{ \"total\": 0 }"), 0755)
if err != nil {
return fmt.Errorf("failed to write usage for '" + project_dir + "'; %w", err)
}

return nil
}
203 changes: 203 additions & 0 deletions create_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,203 @@
package main

import (
"testing"
"path/filepath"
"fmt"
"os/user"
"strings"
)

func TestCreateProjectSimple(t *testing.T) {
reg, err := constructMockRegistry()
if err != nil {
t.Fatalf("failed to create the registry; %v", err)
}
globals := newGlobalConfiguration(reg)

self, err := user.Current()
if err != nil {
t.Fatalf("failed to determine the current user; %v", err)
}
globals.Administrators = append(globals.Administrators, self.Username)

project := "foo"
req_string := fmt.Sprintf(`{ "project": "%s"}`, project)
reqname, err := dumpRequest("create_project", req_string)
if err != nil {
t.Fatalf("failed to create 'create_project' request; %v", err)
}

err = createProjectHandler(reqname, &globals)
if err != nil {
t.Fatalf("failed to create project; %v", err)
}

usage, err := readUsage(filepath.Join(reg, project))
if err != nil {
t.Fatalf("failed to read usage file; %v", err)
}
if usage.Total != 0 {
t.Fatalf("usage should be zero for a newly created project; %v", err)
}
}

func TestCreateProjectFailures(t *testing.T) {
reg, err := constructMockRegistry()
if err != nil {
t.Fatalf("failed to create the registry; %v", err)
}
globals := newGlobalConfiguration(reg)

{
project := "foo"
req_string := fmt.Sprintf(`{ "project": "%s"}`, project)
reqname, err := dumpRequest("create_project", req_string)
if err != nil {
t.Fatalf("failed to create 'create_project' request; %v", err)
}

err = createProjectHandler(reqname, &globals)
if err == nil || !strings.Contains(err.Error(), "not authorized") {
t.Fatalf("creation should have failed without no permissions; %v", err)
}
}

self, err := user.Current()
if err != nil {
t.Fatalf("failed to determine the current user; %v", err)
}
globals.Administrators = append(globals.Administrators, self.Username)

{
project := "..foo"
req_string := fmt.Sprintf(`{ "project": "%s"}`, project)
reqname, err := dumpRequest("create_project", req_string)
if err != nil {
t.Fatalf("failed to create 'create_project' request; %v", err)
}

err = createProjectHandler(reqname, &globals)
if err == nil || !strings.Contains(err.Error(), "invalid project name") {
t.Fatalf("creation should have failed with an invalid project; %v", err)
}
}

{
project := "foo"
req_string := fmt.Sprintf(`{ "project": "%s"}`, project)
reqname, err := dumpRequest("create_project", req_string)
if err != nil {
t.Fatalf("failed to create 'create_project' request; %v", err)
}

err = createProjectHandler(reqname, &globals)
if err != nil {
t.Fatalf("project creation failed; %v", err)
}

err = createProjectHandler(reqname, &globals)
if err == nil || !strings.Contains(err.Error(), "already exists") {
t.Fatalf("duplicate project creation should have failed; %v", err)
}
}
}

func TestCreateProjectNewPermissions(t *testing.T) {
reg, err := constructMockRegistry()
if err != nil {
t.Fatalf("failed to create the registry; %v", err)
}
globals := newGlobalConfiguration(reg)

self, err := user.Current()
if err != nil {
t.Fatalf("failed to determine the current user; %v", err)
}
globals.Administrators = append(globals.Administrators, self.Username)

// Checking that owners are respected.
{
project := "indigo_league"
perm_string := `{ "owners": [ "YAY", "NAY" ] }`
req_string := fmt.Sprintf(`{ "project": "%s", "permissions": %s }`, project, perm_string)
reqname, err := dumpRequest("create_project", req_string)
if err != nil {
t.Fatalf("failed to create 'create_project' request; %v", err)
}

err = createProjectHandler(reqname, &globals)
if err != nil {
t.Fatalf("failed to create a project; %v", err)
}

perms, err := readPermissions(filepath.Join(reg, project))
if err != nil {
t.Fatalf("failed to read the permissions; %v", err)
}
if len(perms.Owners) != 2 || perms.Owners[0] != "YAY" || perms.Owners[1] != "NAY" {
t.Fatal("failed to set the owners correctly in the project permissions")
}
if len(perms.Uploaders) != 0 {
t.Fatal("failed to set the uploaders correctly in the project permissions")
}
}

{
project := "johto_journeys"
new_id := "foo"
perm_string := fmt.Sprintf(`{ "uploaders": [ { "id": "%s" } ] }`, new_id)

req_string := fmt.Sprintf(`{ "project": "%s", "permissions": %s }`, project, perm_string)
reqname, err := dumpRequest("create_project", req_string)
if err != nil {
t.Fatalf("failed to create 'create_project' request; %v", err)
}

err = createProjectHandler(reqname, &globals)
if err != nil {
t.Fatalf("failed to create a project; %v", err)
}

perms, err := readPermissions(filepath.Join(reg, project))
if err != nil {
t.Fatalf("failed to read the permissions; %v", err)
}
if len(perms.Owners) != 1 { // switches to the creating user.
t.Fatal("failed to set the owners correctly in the project permissions")
}
if len(perms.Uploaders) != 1 || perms.Uploaders[0].Id != new_id {
t.Fatal("failed to set the uploaders correctly in the project permissions")
}
}

// Check that uploaders in the permissions are validated.
{
project := "battle_frontier"
req_string := fmt.Sprintf(`{ "project": "%s", "permissions": { "uploaders": [{}] } }`, project)
reqname, err := dumpRequest("create_project", req_string)
if err != nil {
t.Fatalf("failed to create 'create_project' request; %v", err)
}

err = createProjectHandler(reqname, &globals)
if err == nil || !strings.Contains(err.Error(), "invalid 'permissions.uploaders'") {
t.Fatalf("expected project creation to fail from invalid 'uploaders'")
}
}

{
project := "sinnoh_league"
perm_string := `{ "uploaders": [ { "id": "argle", "until": "bargle" } ] }`
req_string := fmt.Sprintf(`{ "project": "%s", "permissions": %s }`, project, perm_string)
reqname, err := dumpRequest("create_project", req_string)
if err != nil {
t.Fatalf("failed to create 'create_project' request; %v", err)
}

err = createProjectHandler(reqname, &globals)
if err == nil || !strings.Contains(err.Error(), "invalid 'permissions.uploaders'") {
t.Fatalf("expected project creation to fail from invalid 'uploaders'")
}
}
}
10 changes: 3 additions & 7 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -98,13 +98,7 @@ func main() {
payload := map[string]interface{}{}

if strings.HasPrefix(reqtype, "upload-") {
config, err0 := uploadHandler(reqpath, &globals)
if err0 == nil {
payload["project"] = config.Project
payload["version"] = config.Version
} else {
reportable_err = err0
}
reportable_err = uploadHandler(reqpath, &globals)

} else if strings.HasPrefix(reqtype, "refresh_latest-") {
res, err0 := refreshLatestHandler(reqpath, &globals)
Expand All @@ -130,6 +124,8 @@ func main() {
reportable_err = approveProbationHandler(reqpath, &globals)
} else if strings.HasPrefix(reqtype, "reject_probation-") {
reportable_err = rejectProbationHandler(reqpath, &globals)
} else if strings.HasPrefix(reqtype, "create_project-") {
reportable_err = createProjectHandler(reqpath, &globals)
} else if strings.HasPrefix(reqtype, "delete_project-") {
reportable_err = deleteProjectHandler(reqpath, &globals)
} else if strings.HasPrefix(reqtype, "delete_asset-") {
Expand Down
Loading

0 comments on commit cf896e8

Please sign in to comment.