AA x AAPI communications for AANTS

[MinhxNguyen7]

We need to have a way for AAPI to ping us to check for changes in section enrollment. The previous understanding is that AAPI will send a list of courses that have been updated. We'd likely need an organization secret so that random people can't spam us.

@andrew-wang0, @IsaacNguyen will be principally developing AANTS on our side, and it'd be good if we can start talking about how this will work. I know that your team has other things on their plate, but this has been the most requested feature for a long time, so it would be great if we can get the ball rolling to safely finish before next enrollment.

Related issues: #962, #971

[andrew-wang0]

Sounds good, currently AAPI is able to do a full scrape of websoc every ~5-10 minutes. (We're working on speeding this up, but for now this is the best we have for how often we can update enrollment statuses).

@IsaacNguyen, let me know if you have any ideas so far of how this could be implemented, but one possibility is setting up an endpoint that serves every section with a change in enrollment status that AA could poll.

[andrew-wang0]

Yes, I was thinking that would work too where AAPI could POST the data to AA so that AA wouldn't have to continuously poll.

However, there are a few reasons why I recommended it to be implemented the other way around:

• AAPI is already set up to serve GET requests (and AA to GET data from AAPI). It would be good to maintain this one-way connection rather than mixing them
• As an open source API, it would be beneficial to have the resource not tied specifically to AA and available to the public (or at least available through an API key)
• Polling eliminates the need for securing an endpoint on AA's side. Although a secret key could be used, the possibility of someone sending malicious notifications is a pretty big risk

Here is a rough idea of how a response from AAPI's side could look like:

GET https://anteaterapi.com/v2/rest/enrollmentChanges

{
  "ok": true,
  "data": {
    "courses": [
      {
        "deptCode": "BIO SCI",
        "courseTitle": "DNA TO ORGANISMS",
        "courseNumber": "93",
        "sections": [
          {
            "sectionCode": "12345",
            "maxCapacity": "75",
            "status": {
              "from": "Waitl",
              "to": "OPEN"
            },
            "numCurrentlyEnrolled": {
              "totalEnrolled": "72",
              "sectionEnrolled": ""
            },
            "numRequested": "85",
            "numOnWaitlist": "0",
            "numWaitlistCap": "10"
          }
        ]
      }
    ],
    "updatedAt": "2024-12-07T12:00:00Z"
  }
}

Every section with a change to its enrollment status (and every newly added section) since the last scrape would be included here. AA would have to poll until updatedAt changes, at which point it could parse the new data and send notifications to subscribed users.

Let me know what you guys think.

[MinhxNguyen7]

As an open source API, it would be beneficial to have the resource not tied specifically to AA and available to the public (or at least available through an API key)

I guess that's fair enough. I was thinking that, through a similar mechanism to API keys, we could add a URL to a list of URLs to be pinged when courses change, but that likely takes more combined effort and can induce a lot of load if there are a lot of consumers.

Polling eliminates the need for securing an endpoint on AA's side. Although a secret key could be used, the possibility of someone sending malicious notifications is a pretty big risk.

The ping would just be for a list of courses that changed. We'd still query back the intersection between courses updated and subscribed.

I think, then, going with polling would be the move.

[MinhxNguyen7]

That being said, I think it would be easier if we could just check the enrollment status of every section that people are subscribed to, then we compare the timestamps to see if we want to send a notification on our end.

[andrew-wang0]

Yep, we could make that a query parameter on the endpoint to filter for just the courses/sections that users are subscribed to

[MinhxNguyen7]

Example qeury: https://anteaterapi.com/v2/rest/enrollmentChanges?term=2024-WINTER&sections=123456,234567

[MinhxNguyen7]

@andrew-wang0 is there an issue up on the API side, and what would the timeline be for this?

[andrew-wang0]

Been discussing with @sanskarm7 about implementation details on our end. From the looks of it, we're thinking about a soft deadline of end of Winter quarter.

Will put an issue up ASAP.

[IsaacNguyen]

Hi @andrew-wang0! Adding onto this, is there a possibility you guys could add functionality to detect when a class's major-only restriction is dropped to this endpoint? I'm unsure if something like this exists already in AAPI, but it would be a great feature if our users could also be notified when a class's restrictions are dropped. Thank you!

[andrew-wang0]

Hey @IsaacNguyen, unfortunately I don't believe that there is a universal way to scrape restriction drop timelines. Although it is listed on websoc, the format varies for each department (and sometimes even links to other websites or spreadsheets in order to get the info).

For now, I don't think we're planning on implementing this.

[IsaacNguyen]

Okay that makes sense. Thank you!

[MinhxNguyen7]

The restriction codes are standardized, not the timelines. That's the same as class statuses.

@IsaacNguyen, there are more restrictions than just major restrictions. Click on the letter in the restriction column to see the definitions

[IsaacNguyen]

Besides major restrictions, which other restrictions have deadlines of when they’re dropped? The only ones here that make sense are the restrictions regarding a student’s class (freshman, sophomore, etc.) but I’m honestly unsure if those are dropped. Sorry I’m just not very familiar with how non-major restrictions work.

[MinhxNguyen7]

No problem. We don't know what the policy is, and they all work the sme way, so we can just implement all of them

[IsaacNguyen]

Hi @andrew-wang0! I was wondering what would be outputted from your end if a class has not changed since the last scrape. Would it just show up with whatever data was updated since the last scrape? Also, during the period of time before any classes have any enrollment changes (like before any enrolllment windows have opened), what kind of output would I expect? Would it just be empty? Thank you!

[sanskarm7]

Hey Isaac, Jordan and I are currently working out the details as to how the variety of possible events will be addressed, and I think it'll be more solidified after we are able to meet with the AntAlmanac team as well.

However, I do have an initial idea for how we can do this. Through consistently updating an additional table with only the recently updated information, while still having a normal "daily" update to the enrollment history table, we allow for the notification system to only have to deal with the newest information.

This would mean that a call that you make to get the newest data will only return the changed data. If nothing was changed since the last scrape, nothing will be returned. As such, if enrollment windows haven't been opened, there will be no changes, so it will be an empty output.