Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to VPCs and RDS #814

Draft
wants to merge 60 commits into
base: master
Choose a base branch
from
Draft

Improvements to VPCs and RDS #814

wants to merge 60 commits into from

Commits on Sep 4, 2024

  1. feat: allow disabling the proxy server and customize client server url (

    #781)
    
    * show proxy ip address in deployment info
    
    * add ProxyInstanceCount and ServerURL configs
    
    - `ProxyInstanceCount` will determine how many proxy instances should be
      deployed. Default to 0, can be 0 or 1.
    - `ServerURL` determines the URL to use with the Mattermost client from
      the agent nodes. It takes precedence over the Site URL, proxy IP, etc.
    
    * downgrade elasticsearch dependency
    
    * ProxyInstanceCount default 0
    
    * faq
    
    * feat: add concurrency to configureAndRunAgents
    
    * Better log tracing
    
    * Update docs/faq.md
    
    Co-authored-by: Alejandro García Montoro <[email protected]>
    
    * SAML configuration
    
    * ltkeycloak
    
    * typos
    
    * fixed slog calls
    
    * updated mattermost-realm.json
    
    * saml configuration
    
    * increase errorsChan buffer
    
    * revert: agent concurrency
    
    * docs: updated faq with SiteURL mention
    
    * NewPointer
    
    * refactor terraform app config
    
    * mlog, ctx, worker number
    
    * debug log
    
    * revert saml changes
    
    * make assets
    
    * go mod tidy
    
    * revert back channel to size 2, return early
    
    ---------
    
    Co-authored-by: Alejandro García Montoro <[email protected]>
    fmartingr and agarciamontoro committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    002c986 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d2467c9 View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. [MM-60371] Update Pyroscope to v1 (#793)

    * Update Pyroscope v1
    Add support for goroutine and block profiles
    
    * Unify service name for targets
    
    * Update test
    streamer45 authored and fmartingr committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    1ded5bc View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    dfd403f View commit details
    Browse the repository at this point in the history
  3. MM-56605: Create results file and print to file (#729)

    * MM-56605: Create results file and print to file
    
    * Separate writter into its own function for later
    
    * Mofify writeToFile. Passing the file object.
    
    * implement 	multiWriter so I can double the writes and then send them to cosole as well as file.
    
    * Replace file object with interface.
    
    * - Remove separate function to create file
    - Move file creatuin and writting to the file to Comparison function
    
    * Draft unti test for PrintResults
    
    * Make test runnable by using LoadTests and Status properly.
    
    * - Solve missing use of interface in the initail code chnage
    - Solve whitespace mismatch
    
    * Change expected to allign with the output. This solves failing unit test.
    
    * Address most several of the code review feedback point.
    
    * Use existing file path that is already being provided to the function.
    
    * Address linter complaint about use of `Sprintf` for strings that need no formating.
    
    * Address review comments
    
    * Remove line printing an error as it was redundant.
    
    * Update another if block that redundantly prints and returns the same error.
    
    * Implement missed correction from Alejandro. If block must return the correct var when returning the error.
    Ensure this is not occuring on other lines I've added.
    DHaussermann authored and fmartingr committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    f3bf239 View commit details
    Browse the repository at this point in the history
  4. Restore the saml-idp.crt file in assets (#792)

    Without this line, the deployment fails because
    it cannot do t.getAsset
    agnivade authored and fmartingr committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    1ddbd16 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. fix: updateuserauth

    fmartingr committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    42e42f5 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. Configuration menu
    Copy the full SHA
    155c12e View commit details
    Browse the repository at this point in the history
  2. feat(ltkeycloak): added flag to force migrate all users (#802)

    * ltkeycloak: added flag to force migrate all users
    
    * changed flag
    
    * avoid logging and returning error
    
    * modified comment
    
    * remove duplicated restoreasset
    
    * reverted condition to return early
    
    * check for nil values
    
    * use UpdateUserAuth
    
    * rename variable name
    
    * Update cmd/ltkeycloak/from_mattermost.go
    
    Co-authored-by: Alejandro García Montoro <[email protected]>
    
    ---------
    
    Co-authored-by: Alejandro García Montoro <[email protected]>
    fmartingr and agarciamontoro committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    0497039 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. Setup VPC parameter for all resources and allow custom database engin…

    …e version (#813)
    
    * customizable database engine version
    
    * Use same VPC for all resources
    
    * make assets
    
    * removed engineversion validation
    
    * wait for stdout
    
    * use default subnet if not specified
    fmartingr authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    e4a29f0 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. Configuration menu
    Copy the full SHA
    86222d4 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. Configuration menu
    Copy the full SHA
    7535b0a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7c21c7f View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2c6c729 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    dc68578 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    59020dc View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    e051958 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    d41273e View commit details
    Browse the repository at this point in the history
  8. BRANCH: Quote password

    fmartingr committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    93d5166 View commit details
    Browse the repository at this point in the history
  9. typo

    fmartingr committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    d46c448 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    737fc20 View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    f2c6466 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    ef654a5 View commit details
    Browse the repository at this point in the history

Commits on Oct 8, 2024

  1. Configuration menu
    Copy the full SHA
    e6da243 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    baef7cc View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ef0492a View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b8ecc1a View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    b11850f View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. [MM-61113] Allow multiple customizable subnets (#833)

    * multiple subnet support
    
    * wait for terraform stdout
    
    * revert elasticsearch zone awareness
    
    * fixed line removed on merge conflict
    
    * AWSAvailabilityZone compatibility
    
    * support a single subnet in db groups
    
    * refactor to use explicit subnets
    
    * elasticsearch multiple subnet support
    
    * docs
    
    * review comments
    
    * use list of subnets for all resources
    
    * tags for db-subnet-group
    
    * updated sample files
    
    * aws_db_subnet_group -> aws_elasticcache_subnet_group
    
    * use reflect.DeepEqual to simplify code
    
    * Update docs/config/deployer.md
    
    Co-authored-by: Alejandro García Montoro <[email protected]>
    
    * Update deployment/config.go
    
    Co-authored-by: Alejandro García Montoro <[email protected]>
    
    * docs link to aws docs
    
    * function to struct method
    
    ---------
    
    Co-authored-by: Alejandro García Montoro <[email protected]>
    fmartingr and agarciamontoro committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    862c4e3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e85cc89 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    62f7793 View commit details
    Browse the repository at this point in the history
  4. merge fixes

    fmartingr committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    032b359 View commit details
    Browse the repository at this point in the history
  5. use AppAttachIAMProfile

    fmartingr committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    b3e69d1 View commit details
    Browse the repository at this point in the history
  6. avoid newlines in logs

    fmartingr committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    b733c03 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. Format cluster.tf

    My editor wants to make the Terraform files pretty.
    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    24e7a30 View commit details
    Browse the repository at this point in the history
  2. Fix typo

    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    8c4e3e4 View commit details
    Browse the repository at this point in the history
  3. Remove deleted ElasticSearchSettings.VpcID

    This field does no longer exist, we can remove it from the sample files
    and documentation
    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    b7a4ed2 View commit details
    Browse the repository at this point in the history
  4. String() needs to be in the plain type method set

    The method set of a type does not contain the methods on its pointer.
    For fmt.Sprintf and friends to use it when using the "%s" formatter, we
    need the method to be defined in the plain type.
    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    de13532 View commit details
    Browse the repository at this point in the history
  5. Use the correct type in reflect.MakeSlice

    MakeSlice expects its first argument to be the type of a slice, not the
    type of the underlying elements of a slice; i.e., we need to pass
    `[]string`, not `string`.
    
    Before this commit, we were passing the type of the slice's Elem(),
    which is the type of the underlying elements.
    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    0b92ca1 View commit details
    Browse the repository at this point in the history
  6. Default ClusterSubnetIDs' slices to non-nil slices

    Defaulting the size of each slice to 0, we make sure the slices are not
    nil, but valid empty slices.
    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    eb77bbf View commit details
    Browse the repository at this point in the history
  7. Test ClusterSubnetIDs

    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    2ff58ad View commit details
    Browse the repository at this point in the history
  8. Fix ClusterSubnetIDs.IsAnySet

    Iterate over all fields of the ClusterSubnetIDs struct that are slices
    and actually check at least one is non-nil with length greater than 0
    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    cfb64cf View commit details
    Browse the repository at this point in the history
  9. Check length of subnet lists instead of empty str

    Now that we validate that var.cluster_subnet_ids.xyz's length is larger
    than 0, we allow both:
    1. Unset lists, which are indeed serialized as empty strings. The length
       of an empty string is still 0.
    2. Empty lists, which are serialized as lists, but that are not equal to
       the empty string.
    
    If we only check for the empty string, we lose 2.
    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    9c00922 View commit details
    Browse the repository at this point in the history
  10. Use proper subnets in DB cluster and Redis

    We were checking for the length of the `cluster_subnet_ids` variable,
    not the underlying `redis` or `database` lists, which are the ones we
    later use in these definitions.
    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    7214588 View commit details
    Browse the repository at this point in the history
  11. Get default VPC and use it to define subnets

    If we don't define the VPC, leaving ClusterVPCID empty or unset, the
    data.aws_subnets.selected list ends up being empty, since we filter for
    subnets that are contained in such VPC.
    
    To fix this, we retrieve the default VPC, and use its ID to filter the
    subnets to select if no VPC is explicitly set.
    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    d93f792 View commit details
    Browse the repository at this point in the history
  12. Make the AZ filter on subnets conditional

    Similar to the VPC change, we filter the selected subnets by their
    availability zone *only* if the `AWSAvailabilityZone` setting is
    non-empty.
    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    5aa7518 View commit details
    Browse the repository at this point in the history
  13. Remove subnet_id if not specified

    If the user doesn't specify a subnet for a resource, we don't need to
    pick one for it, we just set the subnet_id to null.
    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    8b725dd View commit details
    Browse the repository at this point in the history
  14. make assets

    agarciamontoro committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    d9e2dee View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    45da4f3 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2024

  1. Apply suggestions from code review

    Subnet groups require 2+ subnets
    
    Co-authored-by: Felipe Martin <[email protected]>
    agarciamontoro and fmartingr authored Oct 24, 2024
    Configuration menu
    Copy the full SHA
    c72700c View commit details
    Browse the repository at this point in the history
  2. make assets

    agarciamontoro committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    e1dba81 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    c278150 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    76b5d00 View commit details
    Browse the repository at this point in the history
  5. make assets

    agarciamontoro committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    15953aa View commit details
    Browse the repository at this point in the history
  6. create dist/latest.tar.gz

    Ubuntu authored and fmartingr committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    2496b65 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    0870690 View commit details
    Browse the repository at this point in the history
  8. allow elasticsearch

    fmartingr committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    b49ab37 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    51d1afc View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    bad542c View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2024

  1. Configuration menu
    Copy the full SHA
    10e97dc View commit details
    Browse the repository at this point in the history