diff --git a/README.md b/README.md index 3407ff31..82cf4bae 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,9 @@ # veda-backend + This project deploys a complete backend for a [SpatioTemporal Asset Catalog](https://stacspec.org/) including a postgres database, a metadata API, and raster tiling API. Veda-backend is a non-forked version of the [eoAPI](https://github.com/developmentseed/eoAPI) demo project. Veda-backend is decoupled from the demo project to selectively incorporate new stable functionality from the fast moving development in eoAPI while providing a continuous baseline for veda-backend users and to support project specific business and deployment logic. The primary tools employed in the [eoAPI demo](https://github.com/developmentseed/eoAPI) and this project are: + - [stac-spec](https://github.com/radiantearth/stac-spec) - [stac-api-spec](https://github.com/radiantearth/stac-api-spec) - [stac-fastapi](https://github.com/stac-utils/stac-fastapi) @@ -11,6 +13,7 @@ The primary tools employed in the [eoAPI demo](https://github.com/developmentsee - [eoapi-cdk](https://github.com/developmentseed/eoapi-cdk/tree/main#eoapi-cdk-constructs) + [radiantearth/stac-browser](https://github.com/radiantearth/stac-browser) ## VEDA backend context + ![architecture diagram](.readme/veda-overview-bw.svg) _Edit this diagram in VS Code using the [Draw.io Integration Extension](https://marketplace.visualstudio.com/items?itemName=hediet.vscode-drawio) and export a new SVG_ @@ -33,12 +36,14 @@ An [.example.env](.example.env) template is supplied for local deployments. If u ### Fetch environment variables using AWS CLI -To retrieve the variables for a stage that has been previously deployed, the secrets manager can be used to quickly populate an .env file with [scripts/sync-env-local.sh](scripts/sync-env-local.sh). +To retrieve the variables for a stage that has been previously deployed, the secrets manager can be used to quickly populate an .env file with [scripts/sync-env-local.sh](scripts/sync-env-local.sh). -``` +```bash ./scripts/sync-env-local.sh ``` + ### Basic environment variables + | Name | Explanation | | --- | --- | | `APP_NAME` | Optional app name used to name stack and resources, defaults to `veda-backend` | @@ -48,6 +53,7 @@ To retrieve the variables for a stage that has been previously deployed, the sec | `VEDA_DB_SNAPSHOT_ID` | **Once used always REQUIRED** Optional RDS snapshot identifier to initialize RDS from a snapshot | ### Advanced configuration + The constructs and applications in this project are configured using pydantic. The settings are defined in config.py files stored alongside the associated construct or application--for example the settings for the RDS PostgreSQL construct are defined in database/infrastructure/config.py. For custom configuration, use environment variables to override the pydantic defaults. | Construct | Env Prefix | Configuration | @@ -64,24 +70,28 @@ The constructs and applications in this project are configured using pydantic. T ### Deploying to the cloud #### Install deployment pre-requisites + - [Node](https://nodejs.org/) - [NVM](https://github.com/nvm-sh/nvm#node-version-manager---) - [jq](https://jqlang.github.io/jq/) (used for exporting environment variable secrets to `.env` in [scripts/sync-env-local.sh](/scripts/sync-env-local.sh)) These can be installed with [homebrew](https://brew.sh/) on MacOS -``` + +```bash brew install node brew install nvm brew install jq ``` #### Virtual environment example -``` + +```bash python3 -m venv .venv source .venv/bin/activate ``` #### Install requirements + ```bash nvm use --lts npm install --location=global aws-cdk @@ -99,7 +109,7 @@ cdk diff # Execute deployment and standby--security changes will require approval for deployment cdk deploy ``` - + ## Deleting the CloudFormation stack If this is a development stack that is safe to delete, you can delete the stack in CloudFormation console or via `cdk destroy`, however, the additional manual steps were required to completely delete the stack resources: @@ -112,24 +122,29 @@ If this is a development stack that is safe to delete, you can delete the stack ## Custom deployments The default settings for this project generate a complete AWS environment including a VPC and gateways for the stack. See this guidance for adjusting the veda-backend stack for existing managed and/or shared AWS environments. + - [Deploy to an existing managed AWS environment](docs/deploying_to_existing_environments.md) - [Creating a shared base VPC and AWS environment](docs/deploying_to_existing_environments.md#optional-deploy-standalone-base-infrastructure) ## Local Docker deployment Start up a local stack -``` + +```bash docker compose up ``` + Clean up after running locally -``` + +```bash docker compose down ``` ## Running tests locally To run tests implicated in CI, a script is included that requires as little setup as possible -``` + +```bash ./scripts/run-local-tests.sh ``` @@ -137,7 +152,7 @@ In case of failure, all container logs will be written out to `container_logs.lo # Operations -## Adding new data to veda-backend +## Adding new data to veda-backend > **Warning** PgSTAC records should be loaded in the database using [pypgstac](https://github.com/stac-utils/pgstac#pypgstac) for proper indexing and partitioning. @@ -145,11 +160,13 @@ The VEDA ecosystem includes tools specifially created for loading PgSTAC records ## Support scripts Support scripts are provided for manual system operations. + - [Rotate pgstac password](support_scripts/README.md#rotate-pgstac-password) # VEDA ecosystem ## Projects + | Name | Explanation | | --- | --- | | **veda-backend** | Central index (database) and APIs for recording, discovering, viewing, and using VEDA assets | @@ -159,6 +176,7 @@ Support scripts are provided for manual system operations. | [**veda-data**](https://github.com/NASA-IMPACT/veda-data) | Collection and asset discovery configuration | | [**veda-data-airflow**](https://github.com/NASA-IMPACT/veda-data-airflow) | Cloud optimize data assets and submit records for publication to veda-stac-ingestor | | [**veda-docs**](https://github.com/NASA-IMPACT/veda-docs) | Documentation repository for end users of VEDA ecosystem data and tools | +| [**veda-routes**](https://github.com/NASA-IMPACT/veda-routes)| Configuration for VEDA's Content Delivery Network | ## VEDA usage examples @@ -169,7 +187,9 @@ Support scripts are provided for manual system operations. # STAC community resources ## STAC browser + Radiant Earth's [stac-browser](https://github.com/radiantearth/stac-browser) is a browser for STAC catalogs. The demo version of this browser [radiantearth.github.io/stac-browser](https://radiantearth.github.io/stac-browser/#/) can be used to browse the contents of the veda-backend STAC catalog, paste the veda-backend stac-api URL deployed by this project in the demo and click load. Read more about the recent developments and usage of stac-browser [here](https://medium.com/radiant-earth-insights/the-exciting-future-of-the-stac-browser-2351143aa24b). # License + This project is licensed under **Apache 2**, see the [LICENSE](LICENSE) file for more details. diff --git a/app.py b/app.py index c2bf290a..8c28038b 100644 --- a/app.py +++ b/app.py @@ -8,14 +8,12 @@ from config import veda_app_settings from database.infrastructure.construct import RdsConstruct -from domain.infrastructure.construct import DomainConstruct from ingest_api.infrastructure.config import IngestorConfig as ingest_config from ingest_api.infrastructure.construct import ApiConstruct as ingest_api_construct from ingest_api.infrastructure.construct import IngestorConstruct as ingestor_construct from network.infrastructure.construct import VpcConstruct from permissions_boundary.infrastructure.construct import PermissionsBoundaryAspect from raster_api.infrastructure.construct import RasterApiLambdaConstruct -from routes.infrastructure.construct import CloudfrontDistributionConstruct from s3_website.infrastructure.construct import VedaWebsite from stac_api.infrastructure.construct import StacApiLambdaConstruct @@ -71,15 +69,12 @@ def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: stage=veda_app_settings.stage_name(), ) -domain = DomainConstruct(veda_stack, "domain", stage=veda_app_settings.stage_name()) - raster_api = RasterApiLambdaConstruct( veda_stack, "raster-api", stage=veda_app_settings.stage_name(), vpc=vpc.vpc, database=database, - domain=domain, ) stac_api = StacApiLambdaConstruct( @@ -89,23 +84,12 @@ def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: vpc=vpc.vpc, database=database, raster_api=raster_api, - domain=domain, ) website = VedaWebsite( veda_stack, "stac-browser-bucket", stage=veda_app_settings.stage_name() ) -veda_routes = CloudfrontDistributionConstruct( - veda_stack, - "routes", - stage=veda_app_settings.stage_name(), - raster_api_id=raster_api.raster_api.api_id, - stac_api_id=stac_api.stac_api.api_id, - origin_bucket=website.bucket, - region=veda_app_settings.cdk_default_region, -) - # Only create a stac browser if we can infer the catalog url from configuration before synthesis (API Gateway URL not yet available) stac_catalog_url = veda_app_settings.get_stac_catalog_url() if stac_catalog_url: @@ -128,7 +112,6 @@ def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: raster_api_url=raster_api.raster_api.url, ) - ingest_api = ingest_api_construct( veda_stack, "ingest-api", @@ -136,7 +119,6 @@ def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: db_secret=database.pgstac.secret, db_vpc=vpc.vpc, db_vpc_subnets=database.vpc_subnets, - domain=domain, ) ingestor = ingestor_construct( @@ -149,51 +131,6 @@ def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None: db_vpc_subnets=database.vpc_subnets, ) -veda_routes.add_ingest_behavior( - ingest_api=ingest_api.api, stage=veda_app_settings.stage_name() -) - -# Must be done after all CF behaviors exist -veda_routes.create_route_records(stage=veda_app_settings.stage_name()) - - -# TODO this conditional supports deploying a second set of APIs to a separate custom domain and should be removed if no longer necessary -if veda_app_settings.alt_domain(): - alt_domain = DomainConstruct( - veda_stack, - "alt-domain", - stage=veda_app_settings.stage_name(), - alt_domain=True, - ) - - alt_raster_api = RasterApiLambdaConstruct( - veda_stack, - "alt-raster-api", - stage=veda_app_settings.stage_name(), - vpc=vpc.vpc, - database=database, - domain_name=alt_domain.raster_domain_name, - ) - - alt_stac_api = StacApiLambdaConstruct( - veda_stack, - "alt-stac-api", - stage=veda_app_settings.stage_name(), - vpc=vpc.vpc, - database=database, - raster_api=raster_api, - domain_name=alt_domain.stac_domain_name, - ) - - alt_ingest_api = ingest_api_construct( - veda_stack, - "alt-ingest-api", - config=ingestor_config, - db_secret=database.pgstac.secret, - db_vpc=vpc.vpc, - domain_name=alt_domain.ingest_domain_name, - ) - git_sha = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode().strip() try: git_tag = subprocess.check_output(["git", "describe", "--tags"]).decode().strip() diff --git a/config.py b/config.py index 091d6a38..c792b1b7 100644 --- a/config.py +++ b/config.py @@ -90,7 +90,12 @@ class vedaAppSettings(BaseSettings): veda_stac_root_path: str = Field( "", - description="Optional path prefix to add to all api endpoints. Used to infer url of stac-api before app synthesis.", + description="STAC API root path. Used to infer url of stac-api before app synthesis.", + ) + + veda_raster_root_path: str = Field( + "", + description="Raster API root path", ) veda_domain_create_custom_subdomains: bool = Field( @@ -115,15 +120,6 @@ def cdk_env(self) -> dict: else: return {} - def alt_domain(self) -> bool: - """True if alternative domain and host parameters provided""" - return all( - [ - self.veda_domain_alt_hosted_zone_id, - self.veda_domain_alt_hosted_zone_name, - ] - ) - def stage_name(self) -> str: """Force lowercase stage name""" return self.stage.lower() diff --git a/domain/infrastructure/config.py b/domain/infrastructure/config.py deleted file mode 100644 index b74c4e88..00000000 --- a/domain/infrastructure/config.py +++ /dev/null @@ -1,49 +0,0 @@ -"""Configuration options for a custom API domain.""" - -from typing import Optional - -from pydantic import BaseSettings, Field - - -class vedaDomainSettings(BaseSettings): - """Application settings""" - - hosted_zone_id: Optional[str] = Field( - None, description="Route53 hosted zone identifier if using a custom domain name" - ) - hosted_zone_name: Optional[str] = Field( - None, description="Custom domain name, i.e. veda-backend.xyz" - ) - create_custom_subdomains: bool = Field( - False, - description=( - "When true and hosted zone config is provided, create a unique subdomain for stac and raster apis. " - "For example -stac. and -raster." - ), - ) - api_prefix: Optional[str] = Field( - None, - description=( - "Domain prefix override supports using a custom prefix instead of the " - "STAGE variabe (an alternate version of the stack can be deployed with a " - "unique STAGE=altprod and after testing prod API traffic can be cut over " - "to the alternate version of the stack by setting the prefix to prod)" - ), - ) - - # Temporary support for deploying APIs to a second custom domain - alt_hosted_zone_id: Optional[str] = Field( - None, description="Second Route53 zone identifier if using a custom domain name" - ) - alt_hosted_zone_name: Optional[str] = Field( - None, description="Second custom domain name, i.e. alt-veda-backend.xyz" - ) - - class Config: - """model config""" - - env_file = ".env" - env_prefix = "VEDA_DOMAIN_" - - -veda_domain_settings = vedaDomainSettings() diff --git a/domain/infrastructure/construct.py b/domain/infrastructure/construct.py deleted file mode 100644 index 61b133fa..00000000 --- a/domain/infrastructure/construct.py +++ /dev/null @@ -1,148 +0,0 @@ -"""CDK Construct for a custom API domain.""" -from typing import Optional - -from aws_cdk import ( - CfnOutput, - aws_apigatewayv2_alpha, - aws_certificatemanager, - aws_route53, - aws_route53_targets, -) -from constructs import Construct - -from .config import veda_domain_settings - - -class DomainConstruct(Construct): - """CDK Construct for a custom API domain.""" - - def __init__( - self, - scope: Construct, - construct_id: str, - stage: str, - alt_domain: Optional[bool] = False, - **kwargs, - ) -> None: - """.""" - super().__init__(scope, construct_id, **kwargs) - - self.stac_domain_name = None - self.raster_domain_name = None - self.ingest_domain_name = None - - if veda_domain_settings.create_custom_subdomains: - # If alternative custom domain provided, use it instead of the default - if alt_domain is True: - hosted_zone_name = veda_domain_settings.alt_hosted_zone_name - hosted_zone_id = veda_domain_settings.alt_hosted_zone_id - else: - hosted_zone_name = veda_domain_settings.hosted_zone_name - hosted_zone_id = veda_domain_settings.hosted_zone_id - - hosted_zone = aws_route53.HostedZone.from_hosted_zone_attributes( - self, - "hosted-zone", - hosted_zone_id=hosted_zone_id, - zone_name=hosted_zone_name, - ) - certificate = aws_certificatemanager.Certificate( - self, - "certificate", - domain_name=f"*.{hosted_zone_name}", - validation=aws_certificatemanager.CertificateValidation.from_dns( - hosted_zone=hosted_zone - ), - ) - - # Use custom api prefix if provided or deployment stage if not - if veda_domain_settings.api_prefix: - raster_url_prefix = f"{veda_domain_settings.api_prefix.lower()}-raster" - stac_url_prefix = f"{veda_domain_settings.api_prefix.lower()}-stac" - ingest_url_prefix = f"{veda_domain_settings.api_prefix.lower()}-ingest" - else: - raster_url_prefix = f"{stage.lower()}-raster" - stac_url_prefix = f"{stage.lower()}-stac" - ingest_url_prefix = f"{stage.lower()}-ingest" - raster_domain_name = f"{raster_url_prefix}.{hosted_zone_name}" - stac_domain_name = f"{stac_url_prefix}.{hosted_zone_name}" - ingest_domain_name = f"{ingest_url_prefix}.{hosted_zone_name}" - - self.raster_domain_name = aws_apigatewayv2_alpha.DomainName( - self, - "rasterApiCustomDomain", - domain_name=raster_domain_name, - certificate=certificate, - ) - - aws_route53.ARecord( - self, - "raster-api-dns-record", - zone=hosted_zone, - target=aws_route53.RecordTarget.from_alias( - aws_route53_targets.ApiGatewayv2DomainProperties( - regional_domain_name=self.raster_domain_name.regional_domain_name, - regional_hosted_zone_id=self.raster_domain_name.regional_hosted_zone_id, - ) - ), - # Note: CDK will append the hosted zone name (eg: `veda-backend.xyz` to this record name) - record_name=raster_url_prefix, - ) - - self.stac_domain_name = aws_apigatewayv2_alpha.DomainName( - self, - "stacApiCustomDomain", - domain_name=stac_domain_name, - certificate=certificate, - ) - - aws_route53.ARecord( - self, - "stac-api-dns-record", - zone=hosted_zone, - target=aws_route53.RecordTarget.from_alias( - aws_route53_targets.ApiGatewayv2DomainProperties( - regional_domain_name=self.stac_domain_name.regional_domain_name, - regional_hosted_zone_id=self.stac_domain_name.regional_hosted_zone_id, - ) - ), - # Note: CDK will append the hosted zone name (eg: `veda-backend.xyz` to this record name) - record_name=stac_url_prefix, - ) - - self.ingest_domain_name = aws_apigatewayv2_alpha.DomainName( - self, - "ingestApiCustomDomain", - domain_name=ingest_domain_name, - certificate=certificate, - ) - - aws_route53.ARecord( - self, - "ingest-api-dns-record", - zone=hosted_zone, - target=aws_route53.RecordTarget.from_alias( - aws_route53_targets.ApiGatewayv2DomainProperties( - regional_domain_name=self.ingest_domain_name.regional_domain_name, - regional_hosted_zone_id=self.ingest_domain_name.regional_hosted_zone_id, - ) - ), - # Note: CDK will append the hosted zone name (eg: `veda-backend.xyz` to this record name) - record_name=ingest_url_prefix, - ) - - CfnOutput( - self, - "raster-api", - value=f"https://{raster_url_prefix}.{hosted_zone_name}/docs", - ) - CfnOutput( - self, - "stac-api", - value=f"https://{stac_url_prefix}.{hosted_zone_name}/", - ) - CfnOutput( - self, - "ingest-api", - value=f"https://{ingest_url_prefix}.{hosted_zone_name}/", - ) diff --git a/ingest_api/infrastructure/config.py b/ingest_api/infrastructure/config.py index 771d346d..d8277013 100644 --- a/ingest_api/infrastructure/config.py +++ b/ingest_api/infrastructure/config.py @@ -59,19 +59,35 @@ class IngestorConfig(BaseSettings): description="Set optional global parameter to 'requester' if the requester agrees to pay S3 transfer costs", ) - stac_api_url: str = Field(description="URL of STAC API used to serve STAC Items") - - raster_api_url: str = Field( - description="URL of Raster API used to serve asset tiles" - ) - ingest_root_path: str = Field("", description="Root path for ingest API") - custom_host: Optional[str] = Field(description="Custom host name") db_pgstac_version: str = Field( ..., description="Version of PgStac database, i.e. 0.5", ) + stac_api_url: str = Field( + description="URL of STAC API Gateway endpoint used to serve STAC Items" + ) + + raster_api_url: str = Field( + description="URL of Raster API Gateway endpoing used to serve asset tiles" + ) + + custom_host: Optional[str] = Field( + None, + description="Complete url of custom host including subdomain. Used to infer url of apis before app synthesis.", + ) + + stac_root_path: Optional[str] = Field( + "", + description="STAC API root path. Used to infer url of stac-api before app synthesis.", + ) + + raster_root_path: Optional[str] = Field( + "", + description="Raster API root path. Used to infer url of raster-api before app synthesis.", + ) + class Config: case_sensitive = False env_file = ".env" @@ -87,3 +103,17 @@ def env(self) -> aws_cdk.Environment: account=self.aws_account, region=self.aws_region, ) + + @property + def veda_stac_api_cf_url(self) -> str: + """inferred cloudfront url of the stac api if app is configured with a custom host and root path""" + if self.custom_host and self.stac_root_path: + return f"https://{self.custom_host}{self.stac_root_path}" + return self.stac_api_url + + @property + def veda_raster_api_cf_url(self) -> str: + """inferred cloudfront url of the raster api if app is configured with a custom host and root path""" + if self.custom_host and self.stac_root_path: + return f"https://{self.custom_host}{self.raster_root_path}" + return self.raster_api_url diff --git a/ingest_api/infrastructure/construct.py b/ingest_api/infrastructure/construct.py index c4322647..d7dcd05f 100644 --- a/ingest_api/infrastructure/construct.py +++ b/ingest_api/infrastructure/construct.py @@ -1,5 +1,4 @@ import os -import typing from typing import Dict, Optional, Union from aws_cdk import CfnOutput, Duration, RemovalPolicy, Stack @@ -17,9 +16,6 @@ from .config import IngestorConfig -if typing.TYPE_CHECKING: - from domain.infrastructure.construct import DomainConstruct - class ApiConstruct(Construct): def __init__( @@ -30,7 +26,6 @@ def __init__( db_secret: secretsmanager.ISecret, db_vpc: ec2.IVpc, db_vpc_subnets=ec2.SubnetSelection, - domain: Optional["DomainConstruct"] = None, **kwargs, ) -> None: super().__init__(scope, construct_id, **kwargs) @@ -50,11 +45,11 @@ def __init__( "DYNAMODB_TABLE": self.table.table_name, "JWKS_URL": self.jwks_url, "NO_PYDANTIC_SSM_SETTINGS": "1", - "STAC_URL": config.stac_api_url, + "STAC_URL": config.veda_stac_api_cf_url, "USERPOOL_ID": config.userpool_id, "CLIENT_ID": config.client_id, "CLIENT_SECRET": config.client_secret, - "RASTER_URL": config.raster_api_url, + "RASTER_URL": config.veda_raster_api_cf_url, "ROOT_PATH": config.ingest_root_path, "STAGE": config.stage, "COGNITO_DOMAIN": config.cognito_domain, @@ -95,7 +90,6 @@ def __init__( self.api: aws_apigatewayv2_alpha.HttpApi = self.build_api( construct_id=construct_id, handler=self.api_lambda, - domain=domain, custom_host=config.custom_host, ) @@ -193,7 +187,6 @@ def build_api( *, construct_id: str, handler: aws_lambda.IFunction, - domain, custom_host: Optional[str], ) -> aws_apigatewayv2_alpha.HttpApi: integration_kwargs = dict(handler=handler) @@ -212,20 +205,12 @@ def build_api( ) ) - domain_mapping = None - # Legacy method to use a custom subdomain for this api (i.e. -ingest..com) - # If using a custom root path and/or a proxy server, do not use a custom subdomain - if domain and domain.ingest_domain_name: - domain_mapping = aws_apigatewayv2_alpha.DomainMappingOptions( - domain_name=domain.ingest_domain_name - ) stack_name = Stack.of(self).stack_name return aws_apigatewayv2_alpha.HttpApi( self, f"{stack_name}-{construct_id}", default_integration=ingest_api_integration, - default_domain_mapping=domain_mapping, ) def build_jwks_url(self, userpool_id: str) -> str: @@ -271,11 +256,11 @@ def __init__( lambda_env = { "DYNAMODB_TABLE": table.table_name, "NO_PYDANTIC_SSM_SETTINGS": "1", - "STAC_URL": config.stac_api_url, + "STAC_URL": config.veda_stac_api_cf_url, "USERPOOL_ID": config.userpool_id, "CLIENT_ID": config.client_id, "CLIENT_SECRET": config.client_secret, - "RASTER_URL": config.raster_api_url, + "RASTER_URL": config.veda_raster_api_cf_url, } if config.raster_data_access_role_arn: diff --git a/raster_api/infrastructure/construct.py b/raster_api/infrastructure/construct.py index c7362148..b75a9f21 100644 --- a/raster_api/infrastructure/construct.py +++ b/raster_api/infrastructure/construct.py @@ -1,8 +1,6 @@ """CDK Constrcut for a Lambda based TiTiler API with pgstac extension.""" import os -import typing -from typing import Optional from aws_cdk import ( CfnOutput, @@ -19,9 +17,6 @@ from .config import veda_raster_settings -if typing.TYPE_CHECKING: - from domain.infrastructure.construct import DomainConstruct - class RasterApiLambdaConstruct(Construct): """CDK Construct for a Lambda based TiTiler API with pgstac extension.""" @@ -34,8 +29,6 @@ def __init__( vpc, database, code_dir: str = "./", - # domain_name: aws_apigatewayv2_alpha.DomainName = None, - domain: Optional["DomainConstruct"] = None, **kwargs, ) -> None: """.""" @@ -102,19 +95,10 @@ def __init__( ) ) - domain_mapping = None - # Legacy method to use a custom subdomain for this api (i.e. -raster..com) - # If using a custom root path and/or a proxy server, do not use a custom subdomain - if domain and domain.raster_domain_name: - domain_mapping = aws_apigatewayv2_alpha.DomainMappingOptions( - domain_name=domain.raster_domain_name - ) - self.raster_api = aws_apigatewayv2_alpha.HttpApi( self, f"{stack_name}-{construct_id}", default_integration=raster_api_integration, - default_domain_mapping=domain_mapping, ) CfnOutput( diff --git a/routes/infrastructure/config.py b/routes/infrastructure/config.py deleted file mode 100755 index 0d2a878a..00000000 --- a/routes/infrastructure/config.py +++ /dev/null @@ -1,64 +0,0 @@ -"""Settings for Cloudfront distribution - any environment variables starting with -`VEDA_` will overwrite the values of variables in this file -""" -from typing import Optional - -from pydantic import BaseSettings, Field - - -class vedaRouteSettings(BaseSettings): - """Veda Route settings""" - - cloudfront: Optional[bool] = Field( - False, - description="Boolean if Cloudfront Distribution should be deployed", - ) - - cloudfront_oac: Optional[bool] = Field( - True, - description="Boolean that configures Cloufront STAC Browser Origin with Origin Access Control", - ) - - # STAC S3 browser bucket name - stac_browser_bucket: Optional[str] = Field( - None, description="STAC browser S3 bucket name" - ) - - # API Gateway URLs - ingest_url: Optional[str] = Field( - "", - description="URL of ingest API", - ) - - domain_hosted_zone_name: Optional[str] = Field( - None, - description="Domain name for the cloudfront distribution", - ) - - domain_hosted_zone_id: Optional[str] = Field( - None, description="Domain ID for the cloudfront distribution" - ) - - cert_arn: Optional[str] = Field( - None, - description="Certificate’s ARN", - ) - - shared_web_acl_id: Optional[str] = Field( - None, description="Shared Web ACL ID ARN for CloudFront Distribution" - ) - - custom_host: str = Field( - None, - description="Complete url of custom host including subdomain. Used to infer url of stac-api before app synthesis.", - ) - - class Config: - """model config""" - - env_prefix = "VEDA_" - case_sentive = False - env_file = ".env" - - -veda_route_settings = vedaRouteSettings() diff --git a/routes/infrastructure/construct.py b/routes/infrastructure/construct.py deleted file mode 100755 index 3ef67622..00000000 --- a/routes/infrastructure/construct.py +++ /dev/null @@ -1,206 +0,0 @@ -"""CDK Construct for a Cloudfront Distribution.""" -from typing import Optional - -from aws_cdk import CfnOutput, Stack -from aws_cdk import aws_certificatemanager as certificatemanager -from aws_cdk import aws_cloudfront as cf -from aws_cdk import aws_cloudfront_origins as origins -from aws_cdk import aws_iam as iam -from aws_cdk import aws_route53, aws_route53_targets -from aws_cdk import aws_s3 as s3 -from constructs import Construct - -from .config import veda_route_settings - - -class CloudfrontDistributionConstruct(Construct): - """CDK Construct for a Cloudfront Distribution.""" - - def __init__( - self, - scope: Construct, - construct_id: str, - stage: str, - raster_api_id: str, - stac_api_id: str, - origin_bucket: s3.Bucket, - region: Optional[str], - **kwargs, - ) -> None: - """.""" - super().__init__(scope, construct_id) - - stack_name = Stack.of(self).stack_name - - if veda_route_settings.cloudfront: - # Certificate must be in zone us-east-1 - domain_cert = ( - certificatemanager.Certificate.from_certificate_arn( - self, "domainCert", veda_route_settings.cert_arn - ) - if veda_route_settings.cert_arn - else None - ) - - if veda_route_settings.cloudfront_oac: - # create the origin access control resource - cfn_origin_access_control = cf.CfnOriginAccessControl( - self, - "VedaCfnOriginAccessControl", - origin_access_control_config=cf.CfnOriginAccessControl.OriginAccessControlConfigProperty( - name=f"veda-{stage}-oac", - origin_access_control_origin_type="s3", - signing_behavior="always", - signing_protocol="sigv4", - description="Origin Access Control for STAC Browser", - ), - ) - if ( - veda_route_settings.domain_hosted_zone_name - == veda_route_settings.custom_host - ): - self.cf_domain_names = [ - f"{stage}.{veda_route_settings.domain_hosted_zone_name}", - f"{veda_route_settings.domain_hosted_zone_name}", - ] - else: - self.cf_domain_names = [ - f"{stage}.{veda_route_settings.domain_hosted_zone_name}" - ] - - self.distribution = cf.Distribution( - self, - stack_name, - comment=stack_name, - default_behavior=cf.BehaviorOptions( - origin=origins.S3Origin( - origin_bucket, origin_id="stac-browser" - ), - cache_policy=cf.CachePolicy.CACHING_DISABLED, - origin_request_policy=cf.OriginRequestPolicy.CORS_S3_ORIGIN, - response_headers_policy=cf.ResponseHeadersPolicy.CORS_ALLOW_ALL_ORIGINS, - viewer_protocol_policy=cf.ViewerProtocolPolicy.REDIRECT_TO_HTTPS, - ), - certificate=domain_cert, - default_root_object="index.html", - enable_logging=True, - web_acl_id=veda_route_settings.shared_web_acl_id, - domain_names=self.cf_domain_names - if veda_route_settings.domain_hosted_zone_name - else None, - ) - # associate the created OAC with the distribution - distribution_props = self.distribution.node.default_child - if distribution_props is not None: - distribution_props.add_override( - "Properties.DistributionConfig.Origins.0.S3OriginConfig.OriginAccessIdentity", - "", - ) - distribution_props.add_property_override( - "DistributionConfig.Origins.0.OriginAccessControlId", - cfn_origin_access_control.ref, - ) - - # remove the OAI reference from the distribution - all_distribution_props = self.distribution.node.find_all() - for child in all_distribution_props: - if child.node.id == "S3Origin": - child.node.try_remove_child("Resource") - else: - self.distribution = cf.Distribution( - self, - stack_name, - comment=stack_name, - default_behavior=cf.BehaviorOptions( - origin=origins.HttpOrigin( - origin_bucket.bucket_website_domain_name, - protocol_policy=cf.OriginProtocolPolicy.HTTP_ONLY, - origin_id="stac-browser", - ), - cache_policy=cf.CachePolicy.CACHING_DISABLED, - ), - certificate=domain_cert, - default_root_object="index.html", - enable_logging=True, - domain_names=self.cf_domain_names - if veda_route_settings.domain_hosted_zone_name - else None, - ) - - self.distribution.add_behavior( - path_pattern="/api/stac*", - origin=origins.HttpOrigin( - f"{stac_api_id}.execute-api.{region}.amazonaws.com", - origin_id="stac-api", - ), - cache_policy=cf.CachePolicy.CACHING_DISABLED, - allowed_methods=cf.AllowedMethods.ALLOW_ALL, - origin_request_policy=cf.OriginRequestPolicy.ALL_VIEWER_EXCEPT_HOST_HEADER, - ) - - self.distribution.add_behavior( - path_pattern="/api/raster*", - origin=origins.HttpOrigin( - f"{raster_api_id}.execute-api.{region}.amazonaws.com", - origin_id="raster-api", - ), - cache_policy=cf.CachePolicy.CACHING_DISABLED, - allowed_methods=cf.AllowedMethods.ALLOW_ALL, - origin_request_policy=cf.OriginRequestPolicy.ALL_VIEWER_EXCEPT_HOST_HEADER, - ) - - self.hosted_zone = aws_route53.HostedZone.from_hosted_zone_attributes( - self, - "hosted-zone", - hosted_zone_id=veda_route_settings.domain_hosted_zone_id, - zone_name=veda_route_settings.domain_hosted_zone_name, - ) - - # Infer cloudfront arn to add to bucket resource policy - self.distribution_arn = f"arn:aws:cloudfront::{self.distribution.env.account}:distribution/{self.distribution.distribution_id}" - origin_bucket.add_to_resource_policy( - permission=iam.PolicyStatement( - actions=["s3:GetObject"], - conditions={ - "StringEquals": {"aws:SourceArn": self.distribution_arn} - }, - effect=iam.Effect("ALLOW"), - principals=[iam.ServicePrincipal("cloudfront.amazonaws.com")], - resources=[origin_bucket.arn_for_objects("*")], - sid="AllowCloudFrontServicePrincipal", - ) - ) - - CfnOutput(self, "Endpoint", value=self.distribution.domain_name) - - def add_ingest_behavior( - self, - ingest_api, - stage: str, - region: Optional[str] = "us-west-2", - ): - """Required as second step as ingest API depends on stac API route""" - if veda_route_settings.cloudfront: - self.distribution.add_behavior( - "/api/ingest*", - origin=origins.HttpOrigin( - f"{ingest_api.api_id}.execute-api.{region}.amazonaws.com", - origin_id="ingest-api", - ), - cache_policy=cf.CachePolicy.CACHING_DISABLED, - allowed_methods=cf.AllowedMethods.ALLOW_ALL, - origin_request_policy=cf.OriginRequestPolicy.ALL_VIEWER_EXCEPT_HOST_HEADER, - ) - - def create_route_records(self, stage: str): - """This is a seperate function so that it can be called after all behaviors are instantiated""" - if veda_route_settings.cloudfront: - aws_route53.ARecord( - self, - "cloudfront-dns-record", - zone=self.hosted_zone, - target=aws_route53.RecordTarget.from_alias( - aws_route53_targets.CloudFrontTarget(self.distribution) - ), - record_name=stage, - ) diff --git a/stac_api/infrastructure/construct.py b/stac_api/infrastructure/construct.py index f4848a41..c3ca56a0 100644 --- a/stac_api/infrastructure/construct.py +++ b/stac_api/infrastructure/construct.py @@ -1,8 +1,6 @@ """CDK Construct for a Lambda backed API implementing stac-fastapi.""" import os -import typing -from typing import Optional from aws_cdk import ( CfnOutput, @@ -18,9 +16,6 @@ from .config import veda_stac_settings -if typing.TYPE_CHECKING: - from domain.infrastructure.construct import DomainConstruct - class StacApiLambdaConstruct(Construct): """CDK Construct for a Lambda backed API implementing stac-fastapi.""" @@ -34,7 +29,6 @@ def __init__( database, raster_api, # TODO: typing! code_dir: str = "./", - domain: Optional["DomainConstruct"] = None, **kwargs, ) -> None: """.""" @@ -108,19 +102,10 @@ def __init__( ) ) - domain_mapping = None - # Legacy method to use a custom subdomain for this api (i.e. -stac..com) - # If using a custom root path and/or a proxy server, do not use a custom subdomain - if domain and domain.stac_domain_name: - domain_mapping = aws_apigatewayv2_alpha.DomainMappingOptions( - domain_name=domain.stac_domain_name - ) - self.stac_api = aws_apigatewayv2_alpha.HttpApi( self, f"{stack_name}-{construct_id}", default_integration=stac_api_integration, - default_domain_mapping=domain_mapping, ) CfnOutput(