This repository will contain code relevant to Nicholas DiMaria's EE551 Engineering Programming: Python final project.
For this project I created a stock quote tracker. This application will allow a user to enter a symbol, or the name of a company that they are curious about the stock price. The program scrapes https://markets.businessinsider.com for all the information. If a stock ticker is entered, the programs displays the price, how much the price has changed (both percent and dollar amount), as well as the news. The news is displayed with a button below each news header to go to that website. If a company name is entered the search results for that company name are returned in a listbox. The correct company can then be selected from the listbox.
To download this application just clone this repository. Then install the following:
sudo apt-get install python3-lxmlsudo apt-get install python3-pandassudo apt-get install python3-tk
In the main folder just run python3 GUI.py
In the main folder just run python3 main.py
With the programming running perform the following steps:
- Type the stock ticker or company name in the entry box
- Click the "OK" button or hit enter
If entered a stock ticker:
- Click the button below the news article you are interested in
- This will open a browser tab to the news article
If entered a company name:
- Select the company you would like from the listbox
- Click "OK" button or enter again
There is a test file that contains two tests. These tests ensure the following:
- Valid data is returned when passing a stock ticker (specifically "AAPL")
- Checks the company stock ticker returned is "AAPL"
- Checks the company name is "Apple"
- Checks that the price of the stock is greater than 0
- Checks that the company news is a pandas dataframe
- Checks that the "change" price in percent and dollars is something similar to "200.00 (1.10%)"
- A pandas dataframe is returned when a company name is passed (specifically "Apple")
- cd to "Tests" folder
- run
python3 -m pytest tests.py
Click to see details of commits
- Import lxml needs to be installed using sudo apt-get install python3-lxml
- Inspect the elements of the webpage using right click -> inspect
- Using arrow in the top left hand corner of inspect
- Click element you need
- Right click the element and copy the full xpath
- Using tree.xpath([path copied]/text()) and set equal to variable
- Variable will be a list
- Print first element of list
- Allow user to enter stock ticker
- Generate a URL by adding /[stockTicker]-stock to end of https://markets.businessinsider.com/stocks
- Use scraping methods from earlier to get stock prices
- Allows user to enter name of company
- Shows in table form all stock tickers of companies that are returned
- Uses pandas to format datasudo apt-get install python3-pandas
- displaySearch() method found mostly from https://towardsdatascience.com/web-scraping-html-tables-with-python-c9baba21059
- Add main.py file
- Going to be used for testing all the methods on the terminal
- Install tkinter sudo apt-get install python3-tk
- Add GUI file
- Going to house all of the GUI that will call the methods
- Create basic GUI
- When a button is clicked makes call to searchWebsite()
- Changes a label with the data received back from the method
- When the button is clicked take data from the entry box
- Display the returned data in the label
- Add Listbox
- If a panadas dataframe is returned we display the data in a listbox rather than in text
- Want to add the ability to select one item from the list box and click OK to submit
- Simply clears the Listbox when a new stock is searched for
- When “ok” button is clicked we use a try-except statement
- If something in the Listbox is currently selected we do the search on that
- Note: We do string manipulation to get the stock ticker from the end of the string selected
- Otherwise we just search for what is currently in the textbox
- Also add code so that when a stock is searched from the Listbox selection, we replace the words in the entry box
- Change focus on the running of the application to the entry box
- Bind the enter key to the same function as clicking OK
- Created file tests.py
- We run two tests on the search
- First checks that when we type in APPL we get returned “The price of Apple (APPL) is”
- Second checks that if we type “apple” we got a pd dataframe
- Adds some images so that an up or down arrow can be displayed
- Add function upOrDown() in FinalProject.py which scrapes to see if there is an up or down arrow on the page
- Returns true if there is an up arrow or false if there is a down arrow
- In GUI.py we add a new label called image
- We also add up and down arrow images as pgm
- Named upArrow and downArrow
- Call function upOrDown() and display the proper image using image.config(image=[NameOfImage])
- On call to searchWebsite() an object of the company class is now returned
- Returns true if there is an up arrow or false if there is a down arrow
- In GUI.py we add a new label called image
- We also add up and down arrow images as pgm
- Company class contains the stock ticker, the company name and the price
- Soon to add whether the price of the stock is up or down
- In the calls to searchWebsite() we now have to set it to an object
- We then have to call the getData() on the class to print out the data like we did before
- We are combining the upOrDown() method and the searchWebsite()
- This gets all the data at once so that multiple calls aren’t being done
- Had to add another attribute to the company class called up
- Had to account for this when creating a new class
- When we want to use this data we just call company.up
- We now also grab the price change in both dollar amount and percentage
- These are added to the company class as a property call change
- Changed from using pack() to grid()
- Allows configuration of layout using row=X and column =Y
- Remove the pictures of the arrow
- Instead we use text arrows
- Looks nicer and easier to deal with
- Change the color of the text depending on if the price is up or down for the day
- Use fb=’[color]’
- When a stock is search displays news in the listbox
- Want to add that when you click on one you can somehow search?
- Need to fix when you enter a stock that searches website
- Need to fix some of the bad formatting
- We change the attribute changeNumber and changePercent
- Combine back together later
- When they were fetched together the program would fail when a search was done instead
- When we grab the news we now also want the link that it comes from
- In displayNews() method we grab all of the hrefs
- Then we create another column called urls for the dataframe
- Just append the next href to this column every row of the table we loop through
- When iterating through the rows of the dataframe generate a dictionary
- Contains both the title of the articles and the url
- Add a button called search
- Set to disabled when there are search items in the listbox
- Set to enabled when there are news items in the listbox
- When click just prints the url in the terminal for now
- Just adding an icon for the application
- Change the file path of icon
- Update loop for news from 1 to 0 to include all news articles
- Uses webbrowser.open(url,new=new) to launch the browser with the URL of news article that is clicked
- Have to account for the edge case where the article is on their own website
- They usually just point to the file
- We need to append https://markets.businessinsider.com to the beginning of the url and then it goes to the right place
- Just a simple if statement
- Add a global variable called news
- Keeps track of if there is news in the listbox
- If there is news we make the enter button do a search
- Also disable the “OK” button when the news is displayed
- Define a method called callback which is called every time something is changed in the textbox
- We clear the listbox selection
- Re Enable the “OK” button
- Set news to false
- Probably can eliminate the need for all of this we use just use two different listbox (for news and search).
- Just hide them when they need to be
- Have news and searches in different listboxes
- One is hidden and the other one is shown
- Want to make it so the news are labels with buttons beneath
- Instead of a listbox for news, we want labels
- Actually use messages
- Dynamically create labels (and buttons) for each of the news websites
- Arranged in two columns
- As of right now buttons do nothing
- Now add a command to each of the buttons
- Calls search() method
- Uses lambda expression to assign the url needed
- Fix url assignment
- Was off by one
- When we get title of news article and website, also grab the website name
- Store as a separate column in pandas dataframe named “websites”
- We assign this value to the title of the button
- Simply change html.fromstring(page.content) to html.fromstring(page.text)
- Add some padding to the news labels
- Change around stock, ticker, and prices
- Add folders to make application easier to understand
- Had to change some of the imports in code
- Add a few tests for company class
- Add comments for all methods
- Change the wrap size of the news articles
- Update file path in main.py
- Add screenshots for the application for the GitHub page
- Example class contains the main layout for the application
- CustomWidget class contains the widget for displaying stock data
- Label for stock name
- Label for stock price
- Label for stock price change
- Front Page contatins data on DOW, S&P 500, and NASDAQ on page load
- URLs off by one
- Had to fix import for WebScraping file
- Not all stocks have price data on this website
- Account for this by returning "No price data" if none is found in WebScraping
- Have to account for this in the GUI.py
- Just add an if statement to check if company_data.price is "No price data"
- Can then fix formatting accordingly