Put $11,000 Toward Equalify Fixes/Enhancements

[bbertucc]

By April 1, I want to commit $11,000 toward commissioning Equalify-related projects.

Why April?

This isn't an April Fools joke. I am going on paternity leave in April. I want to keep commissioning projects while I am away and bring attention to Equalify. This $11,000 GitHub Issue is a way to accomplish those two goals.

Why $11,000?

I spend about $11k/mo to build Equalify. I also really like the number 11 (see my Notion).

What's the process to get funding?

1. Describe your project as a reply to this issue.
2. Include a total budget to accomplish your project.
3. I'll respond to every reply.
4. If selected, you will be paid 50% on April 1. 50% when you complete your project.*

Will only one project be funded?

Not necessarily. The total budget is $11,000. One project could be funded, or multiple projects. Different projects could be working on the same issue. I think we'll probably fund multiple smaller projects, but I reserve the right to pick one single project.

What is funded?

I'm funding projects with a clear goal by coders with a clear track record.

You get bonus points if you've contributed to Equalify before and/or are based in Louisiana (where R&D credits are sweet!)

Check out existing issues for items there. You don't have to complete an existing issue, but you certainly will get bonus points if you want to.

More info?

I hosted an informational call. We spoke about Equalify architecture, LLMs, and the $11k project. Here is the video for that: https://youtu.be/RBbA4lRnDdc

Any questions?

Please reply to this thread. I'll respond as needed.

*Note: I'll be on paternity leave on April 1, so my feedback will be limited.

[wittjeff]

I have not been able to find an open source video subtitle editor that has a permissive (Apache, BSD, or MIT) license. I'd like to have one for my own project, but it seems to me that it would be a good feature to have included with Equalify as well. So I'd like to write one, with a React front-end. I could use PHP, Python, or (my preference) Node.js / Express (or multiple options) for the back end. I don't have much of a coding track record to point to, so I propose a payment schedule that is entirely pay-for-delivery:
$500 A React app with react-bootstrap, Material UI, or Adobe Spectrum primary components. App loads videos from ffmpeg. User session follows Equalify's data schema.
$1000 App generates a transcript (text only with paragraph breaks) from OpenAI's Whisper library (model size selectable). Support for self-hosted model or use of OpenAI's cloud API.
$2000 App supports modifying the text and timing of captions phrases (initial values provided by Whisper). Subtitles can be saved as either SRT or WebVTT. Data is saved using existing user account model.
$1000 App supports creation of translated SRT or WebVTT files using Whisper language model.
$2000 App supports text-based Audio Description: creation of separate editable (as above) VTT text track with kind="descriptions"
$1000 App supports diarization and user-modifiable speaker labeling. See https://github.com/MahmoudAshraf97/whisper-diarization
$1000 App support visual identification (separation using spleeter or other currently available package) of non-speech audio, to assist with manual edits needed to conform with Captions quality standards. I envision two audio track strips/sliders: one including speech, one with the speech removed.
$1500 App captures text that appears in the video, and automatically provides links to separate text-only transcript / OCR documents as timed annotations in a Descriptions track.
$1000 (speculative) App supports automatic physical positioning of speakers' phrases to be closer to the speaker in the video frame without being obnoxious (WebVTT format only).

[bbertucc]

open source video subtitle editor

@wittjeff that is an interesting proposition and I will definitely consider it as other ideas come in.

[bbertucc]

NOTE: Moved @beatboxchad and my comments to https://github.com/EqualifyEverything/equalify/discussions/284. @beatboxchad comments were not either a question about the opportunity or a proposal for a web accessibility project. https://github.com/EqualifyEverything/equalify/discussions/284 is a better place for off-topic conversation.

[beatboxchad]

One more try:

Is there budget to pay a senior dev for their time evaluating the code?

[bbertucc]

Is there budget to pay a senior dev for their time evaluating the code?

Love that question @beatboxchad! I don't think so. I planned on reviewing any PRs for Equalify after my paternity leave. It would probably be a waste of time to pitch a project for reviewing code at this point. Equalify cash goes to new features, bug fixes, and hosting right now with me reviewing every line of code that enters Equalify.

[beatboxchad]

No, that's not what I'm asking. I'm not talking about reviewing existing contributions. I am (and have been) asking if there's a discovery budget so that developers with limited time and resources can be compensated for their expertise in consulting on new features and enhancements.

In my years of freelancing in NOLA I've had job interviews where they asked a bunch of detailed technical questions which I earnestly and passionately answered, and then did everything I suggested after specifically not hiring me. I'm trying to point out that that's disrespectful of professionals' time and directly counter to "accessibility".

I have disabilities myself, which is why I struggle working full-time. But I do excellent work, which is why my consulting and spec work gets stolen. This is a common discriminatory practice in the tech industry. The most debilitating and costly instance of this sort of thing in my freelance career happened with someone I met at the coworking space you owned at the time.

There's overhead to any contribution, no matter how experienced of a dev you are. You have to get to know the codebase and prepare a proposal, and that's time & effort that should be compensated. OSS doesn't mean non-commercial. You make your development expenses back with consulting projects, and by refusing to pay for people's time, you're limiting access to that business development. You're probably also limiting your own access to funding, along with the talent you would otherwise attract.

So to reiterate my original comment you moved, I'm excited to participate in a project like this one, but I won't offer my experience and consulting services for free, and neither should anyone you really want to work with. This does entail some risk, which is the capital's to absorb. On balance, the inherent value of what we're building will smooth that over.

This is directly on-topic regarding this "opportunity", please don't obfuscate my comments.

[bbertucc]

I am (and have been) asking if there's a discovery budget so that developers with limited time and resources can be compensated for their expertise in consulting on new features and enhancements.

There isn't a discovery budget for that. I totally understand the concern. I don't recommend anyone post a feature or bug that they are worried about someone else doing. Open Source is beautiful and sometimes super annoying because ideas flow freely. Good question.

[heythisischris]

Hey there! I'm interested in tackling the LLM issue you posted (#279). I've had a fair amount of experience integrating OpenAI to handle similar problems, specifically using Node.js (but I could stick to PHP if need be).

I'm not exactly sure what a full budget would be- it might be nice to have a conversation and learn a bit more about Equalify's current stack. A extremely cursory budget estimate would be about $3,500 for a month-long sprint to accomplish the goal- I'm happy to talk further.

[bbertucc]

I'm interested in tackling the LLM issue you posted (#279).

Yay! That's great @heythisischris! This is an issue with a lot of potential that I was hoping someone would jump on.

It might be nice to have a conversation and learn a bit more about Equalify's current stack.

Sure thing! Does any day at 6PM or 1PM work? I was planning on having an open QA to discuss anything and I can schedule it around you.

Let me know then I'll post a Zoom link you (and anyone else interested) can join. In the interest of openness, I'll also post a video here so anyone can understand the stack. PHP is not mandatory.

A extremely cursory budget estimate would be about $3,500 for a month-long sprint to accomplish the goal

That sounds close to my expectations. I would be curious to hear about more deliverables and any questions you have.

[heythisischris]

Hey @bbertucc, I'm glad to hear we're mostly aligned- today at 6pm would be perfect if you're available to meet then. You can send an invite to my email ([email protected]).

[bbertucc]

today at 6pm would be perfect

Oops! Someone else grabbed that time. Looks like any other day than Friday works. I will also post Zoom info here for anyone to join.

[heythisischris]

today at 6pm would be perfect

Oops! Someone else grabbed that time. Looks like any other day than Friday works. I will also post Zoom info here for anyone to join.

Not a problem, tomorrow 1pm then?

[bbertucc]

today at 6pm would be perfect

Oops! Someone else grabbed that time. Looks like any other day than Friday works. I will also post Zoom info here for anyone to join.

Not a problem, tomorrow 1pm then?

Perfect!

Here is the open Zoom meeting link for 3/13 at 1PM CST:
https://us06web.zoom.us/j/85139062645?pwd=5L6aYQictgyLghH1H7dTnF4nXBaaWQ.1

Meeting ID: 851 3906 2645
Passcode: 615618

[eekbk]

@bbertucc Here is a proposal for #280

Equalify React Frontend Proposal

Summary:

This proposal outlines the modernization of the Equalify.app frontend, transitioning from PHP-rendered views to a React application, aiming to enhance the both the developer and user experience, app performance, and future scalability.

Proposed Solution:

I will develop a SPA (Single-Page Application) using React. This will involve creating reusable React components corresponding to the current PHP views and components, implementing client-side routing, and ensuring seamless state management. The backend will be interacted with through HTTP requests.

Implementation Plan:

1. Set up React Development environment: Use Create React app for ease of development / auto configuration.
2. Component Development: Refactor each PHP view/component into corresponding React components.
3. API Integration: Convert frontend API requests to JavaScript using either the native fetch() API or the Axios library. Ensure that backend endpoints are compatible.
4. State Management: Employ React's state hooks and the Context API for managing state, providing simplicity and reducing complexity.
5. Routing: Use React Router to handle client-side routing, matching the current URL structure as closely as possible to preserve SEO and user bookmarks
6. Authentication and Session Management: Integrate existing Auth0 authentication into the React application, utilizing tokens for session management and securing private routes.
7. Accessibility Compliance: Ensure comprehensive adherence to WCAG 2.1 AA standards throughout the React application development, ensuring an inclusive user experience. Implement this by:

• Extensive use of semantic HTML5 and ARIA attributes.
• Utilization of Bootstrap’s accessible features, optimizing development speed while adhering to high accessibility standards.
• Thorough accessibility testing to identify and rectify any barriers, ensuring universal usability.

8. UI/UX Consistency: Maintain a cohesive interface that aligns with established styles, leveraging existing assets and libraries and updating them as needed to complement React’s architecture.
9. Testing: Incorporate robust testing methodologies:

• Utilize Jest or Vitest for unit and API integration testing.
• Implement end-to-end testing frameworks like Cypress or Playwright.

10. Documentation: Update project README and other documentation to reflect the new technology stack and ensure a smooth transition for devs currently working on project.
11. Deployment / Rollout: A Blue/Green deployment strategy is recommended, deploying the new React frontend alongside the current PHP one. Once the new version is fully tested and ready, switch traffic from the old to the new. This will allow for quick rollbacks and minimal downtime.

Consideration: Using Next.js for the Project

While the current plan favors a standalone React SPA, considering Next.js would offer the following advantages:

• Enhanced SEO and performance via SSR.
• SSG capabilities for optimal load times and hosting flexibility.
• Simplified routing and potential for integrated API routes.

However, potential trade-offs include a steeper learning curve and additional server requirements, possibly complicating the development landscape.

Conclusion:

Maintaining Equalify's developer-friendly ethos, the preference is towards a standalone React application. Yet, openness to adopting Next.js exists, contingent on community consensus and aligned project objectives.

Budget Proposal

Overview

This proposal requests $1,000 to support the transition of Equalify.app from PHP-rendered views to a React-based frontend, encompassing development, testing, documentation, and deployment.

[bbertucc]

A few updates from our call:

• Invited folks to Slack. Anyone can join our Slack at this invite link.
• I explained the payment process a bit more: I will pay via Gusto once I think your project is ready to go.
• Video of our discussion is available here: https://youtu.be/RBbA4lRnDdc

[bbertucc]

I've had a fair amount of experience integrating OpenAI to handle similar problems, specifically using Node.js (but I could stick to PHP if need be).

I'm not exactly sure what a full budget would be- it might be nice to have a conversation and learn a bit more about Equalify's current stack. A extremely cursory budget estimate would be about $3,500 for a month-long sprint to accomplish the goal- I'm happy to talk further.

@heythisischris - Just wanted to re-iterate what I said on our call:

I think the budget is inline with work, especially since you would be handling DevOps to create the hosted service.

I would like to just hear a bit more about the UX. Eg: What would you imagine the API endpoints being? Can you give me a general idea of input/output data?

Once I have a general picture of the project, I feel confident to give you the thumbs up. (Inviting you to Gusto pre-emptively.) Happy to answer any other questions.

[bbertucc]

@bbertucc Here is a proposal for #280

@eekbk - This looks really great!

One comment/question: Equalify's API currently only returns data for views and it doesn't handle many actions. How would you handle events, like adding/removing properties, that haven't been spelled out in the API yet? I can do some work before April 1, but unless somone is jumping on #283, the Equalify API would be quite limited.

After we have a gameplan on dealing with Equalify's currently crappy API, I'll reach out to you for payment info then we can get going.

A Few Notes For Everyone

• @eekbk's proposal is exactly the format of what I was after. It's not mandatory, but Kudos!
• Also, if someone is looking for an issue, I would highly recommend Refactor Equalify's API #283

[heythisischris]

I've had a fair amount of experience integrating OpenAI to handle similar problems, specifically using Node.js (but I could stick to PHP if need be).
I'm not exactly sure what a full budget would be- it might be nice to have a conversation and learn a bit more about Equalify's current stack. A extremely cursory budget estimate would be about $3,500 for a month-long sprint to accomplish the goal- I'm happy to talk further.

@heythisischris - Just wanted to re-iterate what I said on our call:

I think the budget is inline with work, especially since you would be handling DevOps to create the hosted service.

I would like to just hear a bit more about the UX. Eg: What would you imagine the API endpoints being? Can you give me a general idea of input/output data?

Once I have a general picture of the project, I feel confident to give you the thumbs up. (Inviting you to Gusto pre-emptively.) Happy to answer any other questions.

Absolutely @bbertucc - here's my proposal:

Equalify LLM API Proposal

Summary:

I will develop a REST API endpoint which suggests "fixes" to developers looking to complete accessibility action items identified by Equalify. It will served via an AWS Lambda running Node.js. The LLM will either be OpenAI GPT-4 or Anthropic Claude 3 (provided via AWS Bedrock).

Implementation Plan:

1. Set up a Node.js Lambda environment, setup CI/CD: I will create a simple TypeScript Lambda function with a build script that deploys the zip bundle directly to the AWS environment using the AWS CLI. I will also setup CI/CD using GitHub Actions.
2. Enable Lambda Function URL: This will allow public access to the Lambda and help with testing.
3. Connect OpenAI and/or Anthropic Claude: I will obtain access to an LLM and integrate to the Lambda function.
4. Study WGAC and begin sketching schema: I will look at commonly identified actions and attempt to create strictly typed actions in JSON format to feed whatever LLM we decide to proceed with. The idea is to emulate a fine-tuned model by feeding a large amount of strictly-typed accessibility context.
5. Define test cases, test API: I will begin testing the newly created LLM API with test cases to make sure accessibility suggestions look decent. I will also begin recording/reporting results and how accurate/helpful the suggestions are.
6. Begin testing with Equalify frontend: I will connect the Equalify frontend with the new LLM API URL and attempt
7. Ask for feedback from the community: I will ask for validation and feedback from the community.

Rough draft schema

The endpoint will be "https://llm.equalify.com". it will only accept POST requests with the following JSON Body data:

{
"ActionId": "a0c5fcb1-0996-45ed-b19b-37195d06c21b",
"Time": "2024-03-13T03:00:00.000Z",
"Source": "Little Forest",
"URL": "https://edupack.dev/sponsors/",
"Type": "Error",
"Message": "```<title>...</title>```\nCheck that the title element describes the document",
"SuggestionId": null
}

The request must be signed with an Auth0 JWT Token set in the header as such:

{
"Authorization": "Bearer eyJraWQiOiIxMU1acVRwRlQ2c1pONWtTd0taSGhRVFk3Q1JWVVpQVkJqRG95U3lRVExrPSIsImFsZyI6IlJTMjU2In0.eyJzdWIiOiIyMTNiNzVkMC02MGYxLTcwNDktM2RlNC00NDRmMWY5ZGZjMjYiLCJ3ZWJzaX"
}

The response will be a JSON Body with the following format:

{
"ActionId": "a0c5fcb1-0996-45ed-b19b-37195d06c21b",
"SuggestionId": "1f5c0c9f-ee33-4bdb-84e5-ea53caa4a5ba",
"Time": "2024-03-13T04:00:00.000Z",
"Title": "You could rename the title attribute to \"EduPack Sponsors\"",
"Code": "<title>EduPack Sponsors</title>",
"Position": [12,48],
"Completed": false
}

We can store suggestions in a new MySQL table for future reference.

Obviously, this is a very simple sample response, but I intend on having much more complex answers and functionality as I develop- this is simply a starting point.

Budget

The proposed budget for this issue is $3,500. It should take no longer than 1 month to fully develop, deploy, test, and release.

[bbertucc]

@heythisischris - your project is approved. We can discuss progress on #296 and I sent you an invite to Gusto. I will process your first payment ($1,750) as soon as you fill out Gusto docs and the taxman gets back to me about how they want me to label your work (hopefully before the end of the week).

[heythisischris]

@bbertucc Excellent!

[eekbk]

One comment/question: Equalify's API currently only returns data for views and it doesn't handle many actions. How would you handle events, like adding/removing properties, that haven't been spelled out in the API yet? I can do some work before April 1, but unless somone is jumping on #283, the Equalify API would be quite limited.

After we have a gameplan on dealing with Equalify's currently crappy API, I'll reach out to you for payment info then we can get going.

@bbertucc I think it's probably best to use placeholder functions for those events in the frontend. These placeholders will simulate the expected interactions and can be easily swapped out with the actual API calls once #283 is completed.

Another option would be for me to create those backend endpoints myself. BUT that may cause conflicts or inconsistencies, especially if someone begins work on refactoring the backend in a comprehensive way as part of #283.

I'm open to discussing these or other options on how to proceed.

[bbertucc]

@bbertucc I think it's probably best to use placeholder functions for those events in the frontend. These placeholders will simulate the expected interactions and can be easily swapped out with the actual API calls once #283 is completed.

Another option would be for me to create those backend endpoints myself. BUT that may cause conflicts or inconsistencies, especially if someone begins work on refactoring the backend in a comprehensive way as part of #283.

Placeholders sound great @eekbk.

Project approved! I Invited you to the GitHub org and Gusto. Any updates an comments related to the project can go into #298. Anything else can go into Slack.

Note for everyone else, with this project approved, we really need someone to get #283 done right.

[hcwiley]

I'm proposing creating a Chrome Extension so usability testers may easily send the selected code, website url, tag(s), “status”, and message to Equalify. The initial project scope:

• A few (~2-4) meetings with Equalify users who rely on assistive technology to understand their needs and shape the functionality of the extension.
• Build a quick prototype Equalify Chrome Extension
• Test installation of Chrome Extension on a clean machine of mine
• Test installation of Chrome Extension with Equalify 3-5 users and screen record (with their permission) to review pain points
• Get notes from users on how the extension works and what is missing/broken after they've had a day or two to use it.

This first version will send data to a dummy API to just validate that the extension works in the UI/UX and not bombard the actual Equalify API with crap.

The next stages include:

• add authentication
• send selected elements to Equalify API
• Add hotkey to enable easier sending
• Sync with the Equalify team on deeper integration and maintenance strategy

My proposed budget for this work to take place in April when spoken in 11's is $333 * 11 ($3663)

[bbertucc]

I'm proposing creating a Chrome Extension so usability testers may easily send the selected code, website url, tag(s), “status”, and message to Equalify. The initial project scope:

Sweet. Approved @hcwiley. I'll send you an invite to Gusto and I updated #297 with your notes. Feel free to take it from here and let me know what else you need on Slack or in that issue

$2,837 now left in the $11k pot, so to speak.

[bbertucc]

I'm going to go ahead and reserve another $333 for #299. We might need more or less, but I want to make sure we have some cash ready for use testing - especially since #297 is approved.

[heythisischris]

Equalify API Refactor Proposal

Summary:

I will refactor the current PHP-based API into a standards compliant REST API using Node.js (leveraging Fastify as our lightweight HTTP framework).

Implementation Plan:

1. Set up a Node.js Fastify repository which properly communicates with a local build of Equalify.
2. Set up an AWS Lambda function which Equalify can deploy directly to for staging/production usage (using the Fastify AWS Lambda wrapper: https://github.com/fastify/aws-lambda-fastify).
3. Make sure all CI/CD pipes are working
4. Begin mapping out the latest schema provided by @bbertucc into Fastify routes.
5. Validate all endpoints, ensure that we're returning the correct HTTP header response codes.
6. Consider migrating the existing LLM API effort into this effort.
7. Once finished, begin asking the community for help with testing/validating the new API.
8. Begin formal documentation (ideally Readme, GitHub Wiki, Notion, or a full featured self-hosted documentation solution w/ search).
9. Deploy to staging/production once everything looks good, deprecate the PHP API, and provide continued support/maintenance!

Budget

The proposed budget for this issue is $2,222. It should take no longer than 1 month to fully develop, deploy, test, and release.

[bbertucc]

Approved @heythisischris - I also assigned you to #283

And with that, I'm closing this $11k April issue.

More coming in future months.

[bbertucc]

One thing that I forgot is that we'll probably need some kind of project management. I'll pay Jessica Link $222 this month to do that. PM will consist of just asking folks what they've done every week and posting it on Friday as regular "Good News Updates". Feel free to join Slack: https://join.slack.com/t/equalifyapp/shared_invite/zt-1sfbgf0fa-CzIHlbFOs0Ww1iSTK4LQ2w