Application Scanner and Analysis Tool

System Objectives

• Scan careers pages for specific companies and find the most relevant job opens for my profile that have been opened in the last 24 hours.
• Analyze each job opening that is found and compare the job description and role requirements to my resume and cover letter.
  • Give a summary of the job description: Job title, summary of role, summary of role requirements and why I would be good for the job
  • Present possible changes to my resume and cover letter to better fit the job description
  • Provide link to application
• Email a report to my email with the findings of scanning and analyzing on a daily basis

Implementation Details & Tasks

Scanning Functionality

• Two methods of scanning for job openings (probably write in two base classes):
  1. Publicly accessible APIs (ideal)
  2. Scraping careers page with python framework (beautifulsoup)
• Functionality to request or scrape jobs from different companies (probably extensions of Scrapper or Requester classes customized to fit specifics of each companies job site):
  1. Apple
     • Can access jobs through making requests to https://jobs.apple.com/
     • Search: https://jobs.apple.com/api/role/search. Request must be made with X-Apple-CSRF-Token header. Value can be found at link described in next point.
     • Need to make call to https://jobs.apple.com/api/csrfToken to get csrf token before searching for jobs.
  2. Microsoft
  3. Netflix
• Standardized format for how to store information from each company
  • Job Title
  • Job Location
  • Company Description
  • Job Description
  • Roles and Responsibilities
  • Role Requirements
  • Preferred Requirements
• Will this run as a script or will it be an API or application hosted on an external server or cloud?
  • Could use my Raspi for this.
  • Could just use an AWS Ec2 instance

Job Description Analysis

Model Decision: gpt-4o vs. gpt-4o-mini

Core Framework: LangChain vs. DSPy &LangChain

• Calling the model with job descriptions:

  • For V0 I am going to do analysis in two steps:
    1. Pass the entire job description and resume
    2. Pass the entire job description and cover letter
  • There may be a better way to split these calls into different and separate calls to get better analysis using a smaller chunk of the job description and/or profile document.

• Prompt Tuning: Need to determine the best prompt to get a succinct and target response that contains the following things:

  • Summary of job description
  • Rating of how and why I would be a good fit for the position
  • Changes to my resume and cover letter that would improve my profile for the role

  Possible initial prompting strategies:
  

  • Zero-shot: From personal experience of doing this process manually for some time, I can conclude that, for my purposes, simply prompting the model directly without any bootstrapped examples yields a reasonably good response.

    • Pros: This method is easy to implement and does not require any pre-generated demonstrations.
      
    • Cons: However, this method tends to yield inconsistent responses in terms of quality and format.

  • Few-Shot: Given I have a good amount of demonstrations built up from running through this process manually I could use some of these responses for bootsrapping.
    

    • Pros: This would provide better standardization between prompts
      
    • Cons: This method would be more costly per prompt. Because the system responses can vary by a massive degree depending on the job description or resume or cover letter text, testing would be required to determine if/how a constant set of demonstrations added to each prompt will influence results.

  • Prompt Chaining - Seeing as there are multiple comparisons being made for each call to the LM (required/preferred qualifications, job summary, role description ect..) it could make sense to split these comparisons into seperate calls to the LM. However,work would need to go into testing what would be the best way to prompt the system given there is so much information being compared for a single resume/cover letter to job description combo? A possible way of doing this is by breaking a part the job description and comparing each section to the resume/cover letter.

    • Pros: This could allow for a bit more flexibility in how I optimize each indevidual prompt and would make the work to do so much more module. Could potentially look into using DSPy for help for optimization :).
      
    • Cons: This method would certainly be more expensive for a single call method.

  • Evaluation Taxonomy - Need to come up with a standardized set of criteria to grade any given response. Some categories that I could use off the top of my head are:

    • Matching key terms that appear in the job description but do not appear in my resume or cover letter to see if the model is responding with an analysis that includes skills that the job description asks for but I do not have [prbably will be numerical]
    • Does the model include any of the skills that are directly referenced in my resume or cover as things I could improve on based on the job description? [yes/no]
    • Does the model format the generated response correctly. [yes/no]
    • Does the model include all of the job description information correctly (job title, company, job location ect..). [yes/no]
    • The model will be generating a "match number" between the job description and the resume/cover letter. I could use job descriptions for jobs that I have been gotten offers for in the past as a testing set to see if the model gives me a high likelyhood for those jobs. This would be a small testing set but may be valuable if I can avoid overfitting.

• Possible improvement tasks:

  • Look into GPT’s Batch API to cut costs.
  • Would Fine-Tuning [possibly any initial changes to temperature or topP as easy first passes] or Knowledge Distillation help in any way?

Email Report

• Send email though an email system. Seems like Sendgrid has a free plan that doesn’t expire which may be a good place to start.
• Run the system Daily to send a report with newly added jobs. Maybe just a simple cron job.