Chapter05_AdvancedHTMLParsing.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<html>\n",
      "<head>\n",
      "<style>\n",
      ".green{\n",
      "\tcolor:#55ff55;\n",
      "}\n",
      ".red{\n",
      "\tcolor:#ff5555;\n",
      "}\n",
      "#text{\n",
      "\twidth:50%;\n",
      "}\n",
      "</style>\n",
      "</head>\n",
      "<body>\n",
      "<h1>War and Peace</h1>\n",
      "<h2>Chapter 1</h2>\n",
      "<div id=\"text\">\n",
      "\"<span class=\"red\">Well, Prince, so Genoa and Lucca are now just family estates of the\n",
      "Buonapartes. But I warn you, if you don't tell me that this means war,\n",
      "if you still try to defend the infamies and horrors perpetrated by\n",
      "that Antichrist- I really believe he is Antichrist- I will have\n",
      "nothing more to do with you and you are no longer my friend, no longer\n",
      "my 'faithful slave,' as you call yourself! But how do you do? I see\n",
      "I have frightened you- sit down and tell me all the news.</span>\"\n",
      "<p></p>\n",
      "It was in July, 1805, and the speaker was the well-known <span class=\"green\">Anna\n",
      "Pavlovna Scherer</span>, maid of honor and favorite of the <span class=\"green\">Empress Marya\n",
      "Fedorovna</span>. With these words she greeted <span class=\"green\">Prince Vasili Kuragin</span>, a man\n",
      "of high rank and importance, who was the first to arrive at her\n",
      "reception. <span class=\"green\">Anna Pavlovna</span> had had a cough for some days. She was, as\n",
      "she said, suffering from la grippe; grippe being then a new word in\n",
      "<span class=\"green\">St. Petersburg</span>, used only by the elite.\n",
      "<p></p>\n",
      "All her invitations without exception, written in French, and\n",
      "delivered by a scarlet-liveried footman that morning, ran as follows:\n",
      "<p></p>\n",
      "\"<span class=\"red\">If you have nothing better to do, Count [or Prince], and if the\n",
      "prospect of spending an evening with a poor invalid is not too\n",
      "terrible, I shall be very charmed to see you tonight between 7 and 10-\n",
      "Annette Scherer.</span>\"\n",
      "<p></p>\n",
      "\"<span class=\"red\">Heavens! what a virulent attack!</span>\" replied <span class=\"green\">the prince</span>, not in the\n",
      "least disconcerted by this reception. He had just entered, wearing\n",
      "an embroidered court uniform, knee breeches, and shoes, and had\n",
      "stars on his breast and a serene expression on his flat face. He spoke\n",
      "in that refined French in which our grandfathers not only spoke but\n",
      "thought, and with the gentle, patronizing intonation natural to a\n",
      "man of importance who had grown old in society and at court. He went\n",
      "up to <span class=\"green\">Anna Pavlovna</span>, kissed her hand, presenting to her his bald,\n",
      "scented, and shining head, and complacently seated himself on the\n",
      "sofa.\n",
      "<p></p>\n",
      "\"<span class=\"red\">First of all, dear friend, tell me how you are. Set your friend's\n",
      "mind at rest,</span>\" said he without altering his tone, beneath the\n",
      "politeness and affected sympathy of which indifference and even\n",
      "irony could be discerned.\n",
      "<p></p>\n",
      "\"<span class=\"red\">Can one be well while suffering morally? Can one be calm in times\n",
      "like these if one has any feeling?</span>\" said <span class=\"green\">Anna Pavlovna</span>. \"<span class=\"red\">You are\n",
      "staying the whole evening, I hope?</span>\"\n",
      "<p></p>\n",
      "\"<span class=\"red\">And the fete at the English ambassador's? Today is Wednesday. I\n",
      "must put in an appearance there,</span>\" said <span class=\"green\">the prince</span>. \"<span class=\"red\">My daughter is\n",
      "coming for me to take me there.</span>\"\n",
      "<p></p>\n",
      "\"<span class=\"red\">I thought today's fete had been canceled. I confess all these\n",
      "festivities and fireworks are becoming wearisome.</span>\"\n",
      "<p></p>\n",
      "\"<span class=\"red\">If they had known that you wished it, the entertainment would\n",
      "have been put off,</span>\" said <span class=\"green\">the prince</span>, who, like a wound-up clock, by\n",
      "force of habit said things he did not even wish to be believed.\n",
      "<p></p>\n",
      "\"<span class=\"red\">Don't tease! Well, and what has been decided about Novosiltsev's\n",
      "dispatch? You know everything.</span>\"\n",
      "<p></p>\n",
      "\"<span class=\"red\">What can one say about it?</span>\" replied <span class=\"green\">the prince</span> in a cold,\n",
      "listless tone. \"<span class=\"red\">What has been decided? They have decided that\n",
      "Buonaparte has burnt his boats, and I believe that we are ready to\n",
      "burn ours.</span>\"\n",
      "<p></p>\n",
      "<span class=\"green\">Prince Vasili</span> always spoke languidly, like an actor repeating a\n",
      "stale part. <span class=\"green\">Anna Pavlovna</span> Scherer on the contrary, despite her forty\n",
      "years, overflowed with animation and impulsiveness. To be an\n",
      "enthusiast had become her social vocation and, sometimes even when she\n",
      "did not feel like it, she became enthusiastic in order not to\n",
      "disappoint the expectations of those who knew her. The subdued smile\n",
      "which, though it did not suit her faded features, always played\n",
      "round her lips expressed, as in a spoiled child, a continual\n",
      "consciousness of her charming defect, which she neither wished, nor\n",
      "could, nor considered it necessary, to correct.\n",
      "<p></p>\n",
      "In the midst of a conversation on political matters <span class=\"green\">Anna Pavlovna</span>\n",
      "burst out:\n",
      "<p></p>\n",
      "\"<span class=\"red\">Oh, don't speak to me of Austria. Perhaps I don't understand\n",
      "things, but Austria never has wished, and does not wish, for war.\n",
      "She is betraying us! Russia alone must save Europe. Our gracious\n",
      "sovereign recognizes his high vocation and will be true to it. That is\n",
      "the one thing I have faith in! Our good and wonderful sovereign has to\n",
      "perform the noblest role on earth, and he is so virtuous and noble\n",
      "that God will not forsake him. He will fulfill his vocation and\n",
      "crush the hydra of revolution, which has become more terrible than\n",
      "ever in the person of this murderer and villain! We alone must\n",
      "avenge the blood of the just one.... Whom, I ask you, can we rely\n",
      "on?... England with her commercial spirit will not and cannot\n",
      "understand the Emperor Alexander's loftiness of soul. She has\n",
      "refused to evacuate Malta. She wanted to find, and still seeks, some\n",
      "secret motive in our actions. What answer did Novosiltsev get? None.\n",
      "The English have not understood and cannot understand the\n",
      "self-abnegation of our Emperor who wants nothing for himself, but only\n",
      "desires the good of mankind. And what have they promised? Nothing! And\n",
      "what little they have promised they will not perform! Prussia has\n",
      "always declared that Buonaparte is invincible, and that all Europe\n",
      "is powerless before him.... And I don't believe a word that Hardenburg\n",
      "says, or Haugwitz either. This famous Prussian neutrality is just a\n",
      "trap. I have faith only in God and the lofty destiny of our adored\n",
      "monarch. He will save Europe!</span>\"\n",
      "<p></p>\n",
      "She suddenly paused, smiling at her own impetuosity.\n",
      "<p></p>\n",
      "\"<span class=\"red\">I think,</span>\" said <span class=\"green\">the prince</span> with a smile, \"<span class=\"red\">that if you had been\n",
      "sent instead of our dear <span class=\"green\">Wintzingerode</span> you would have captured the\n",
      "<span class=\"green\">King of Prussia</span>'s consent by assault. You are so eloquent. Will you\n",
      "give me a cup of tea?</span>\"\n",
      "<p></p>\n",
      "\"<span class=\"red\">In a moment. A propos,</span>\" she added, becoming calm again, \"<span class=\"red\">I am\n",
      "expecting two very interesting men tonight, <span class=\"green\">le Vicomte de Mortemart</span>,\n",
      "who is connected with the <span class=\"green\">Montmorencys</span> through the <span class=\"green\">Rohans</span>, one of\n",
      "the best French families. He is one of the genuine emigres, the good\n",
      "ones. And also the <span class=\"green\">Abbe Morio</span>. Do you know that profound thinker? He\n",
      "has been received by <span class=\"green\">the Emperor</span>. Had you heard?</span>\"\n",
      "<p></p>\n",
      "\"<span class=\"red\">I shall be delighted to meet them,</span>\" said <span class=\"green\">the prince</span>. \"<span class=\"red\">But tell me,</span>\"\n",
      "he added with studied carelessness as if it had only just occurred\n",
      "to him, though the question he was about to ask was the chief motive\n",
      "of his visit, \"<span class=\"red\">is it true that the Dowager Empress wants Baron Funke\n",
      "to be appointed first secretary at Vienna? The baron by all accounts\n",
      "is a poor creature.</span>\"\n",
      "<p></p>\n",
      "<span class=\"green\">Prince Vasili</span> wished to obtain this post for his son, but others\n",
      "were trying through the <span class=\"green\">Dowager Empress Marya Fedorovna</span> to secure it\n",
      "for <span class=\"green\">the baron</span>.\n",
      "<p></p>\n",
      "<span class=\"green\">Anna Pavlovna</span> almost closed her eyes to indicate that neither she\n",
      "nor anyone else had a right to criticize what <span class=\"green\">the Empress</span> desired or\n",
      "was pleased with.\n",
      "<p></p>\n",
      "\"<span class=\"red\">Baron Funke has been recommended to the Dowager Empress by her\n",
      "sister,</span>\" was all she said, in a dry and mournful tone.\n",
      "<p></p>\n",
      "As she named <span class=\"green\">the Empress</span>, <span class=\"green\">Anna Pavlovna's</span> face suddenly assumed an\n",
      "expression of profound and sincere devotion and respect mingled with\n",
      "sadness, and this occurred every time she mentioned her illustrious\n",
      "patroness. She added that <span class=\"green\">Her Majesty</span> had deigned to show <span class=\"green\">Baron\n",
      "Funke</span>, and again her face clouded over with sadness.\n",
      "<p></p>\n",
      "<span class=\"green\">The prince</span> was silent and looked indifferent. But, with the\n",
      "womanly and courtierlike quickness and tact habitual to her, <span class=\"green\">Anna\n",
      "Pavlovna</span> wished both to rebuke him (for daring to speak he had done of\n",
      "a man recommended to <span class=\"green\">the Empress</span>) and at the same time to console him,\n",
      "so she said:\n",
      "<p></p>\n",
      "\"<span class=\"red\">Now about your family. Do you know that since your daughter came\n",
      "out everyone has been enraptured by her? They say she is amazingly\n",
      "beautiful.</span>\"\n",
      "<p></p>\n",
      "<span class=\"green\">The prince</span> bowed to signify his respect and gratitude.\n",
      "<p></p>\n",
      "\"<span class=\"red\">I often think,</span>\" she continued after a short pause, drawing nearer\n",
      "to the prince and smiling amiably at him as if to show that\n",
      "political and social topics were ended and the time had come for\n",
      "intimate conversation- \"<span class=\"red\">I often think how unfairly sometimes the\n",
      "joys of life are distributed. Why has fate given you two such splendid\n",
      "children? I don't speak of <span class=\"green\">Anatole</span>, your youngest. I don't like\n",
      "him,</span>\" she added in a tone admitting of no rejoinder and raising her\n",
      "eyebrows. \"<span class=\"red\">Two such charming children. And really you appreciate\n",
      "them less than anyone, and so you don't deserve to have them.</span>\"\n",
      "<p></p>\n",
      "And she smiled her ecstatic smile.\n",
      "<p></p>\n",
      "\"<span class=\"red\">I can't help it,</span>\" said <span class=\"green\">the prince</span>. \"<span class=\"red\">Lavater would have said I\n",
      "lack the bump of paternity.</span>\"\n",
      "<p></p>\n",
      "\"<span class=\"red\">Don't joke; I mean to have a serious talk with you. Do you know I\n",
      "am dissatisfied with your younger son? Between ourselves</span>\" (and her\n",
      "face assumed its melancholy expression), \"<span class=\"red\">he was mentioned at Her\n",
      "Majesty's and you were pitied....</span>\"\n",
      "<p></p>\n",
      "<span class=\"green\">The prince</span> answered nothing, but she looked at him significantly,\n",
      "awaiting a reply. He frowned.\n",
      "<p></p>\n",
      "\"<span class=\"red\">What would you have me do?</span>\" he said at last. \"<span class=\"red\">You know I did all\n",
      "a father could for their education, and they have both turned out\n",
      "fools. Hippolyte is at least a quiet fool, but Anatole is an active\n",
      "one. That is the only difference between them.</span>\" He said this smiling\n",
      "in a way more natural and animated than usual, so that the wrinkles\n",
      "round his mouth very clearly revealed something unexpectedly coarse\n",
      "and unpleasant.\n",
      "<p></p>\n",
      "\"<span class=\"red\">And why are children born to such men as you? If you were not a\n",
      "father there would be nothing I could reproach you with,</span>\" said <span class=\"green\">Anna\n",
      "Pavlovna</span>, looking up pensively.\n",
      "<p></p>\n",
      "\"<span class=\"red\">I am your faithful slave and to you alone I can confess that my\n",
      "children are the bane of my life. It is the cross I have to bear. That\n",
      "is how I explain it to myself. It can't be helped!</span>\"\n",
      "<p></p>\n",
      "He said no more, but expressed his resignation to cruel fate by a\n",
      "gesture. <span class=\"green\">Anna Pavlovna</span> meditated.\n",
      "</div>\n",
      "</body>\n",
      "</html>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from urllib.request import urlopen\n",
    "from bs4 import BeautifulSoup\n",
    "html = urlopen('http://www.pythonscraping.com/pages/warandpeace.html')\n",
    "bs = BeautifulSoup(html, 'html.parser')\n",
    "print(bs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from urllib.request import urlopen\n",
    "from bs4 import BeautifulSoup\n",
    "html = urlopen('http://www.pythonscraping.com/pages/warandpeace.html')\n",
    "bs = BeautifulSoup(html, \"html.parser\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Well, Prince, so Genoa and Lucca are now just family estates of the\n",
      "Buonapartes. But I warn you, if you don't tell me that this means war,\n",
      "if you still try to defend the infamies and horrors perpetrated by\n",
      "that Antichrist- I really believe he is Antichrist- I will have\n",
      "nothing more to do with you and you are no longer my friend, no longer\n",
      "my 'faithful slave,' as you call yourself! But how do you do? I see\n",
      "I have frightened you- sit down and tell me all the news.\n",
      "Anna\n",
      "Pavlovna Scherer\n",
      "Empress Marya\n",
      "Fedorovna\n",
      "Prince Vasili Kuragin\n",
      "Anna Pavlovna\n",
      "St. Petersburg\n",
      "If you have nothing better to do, Count [or Prince], and if the\n",
      "prospect of spending an evening with a poor invalid is not too\n",
      "terrible, I shall be very charmed to see you tonight between 7 and 10-\n",
      "Annette Scherer.\n",
      "Heavens! what a virulent attack!\n",
      "the prince\n",
      "Anna Pavlovna\n",
      "First of all, dear friend, tell me how you are. Set your friend's\n",
      "mind at rest,\n",
      "Can one be well while suffering morally? Can one be calm in times\n",
      "like these if one has any feeling?\n",
      "Anna Pavlovna\n",
      "You are\n",
      "staying the whole evening, I hope?\n",
      "And the fete at the English ambassador's? Today is Wednesday. I\n",
      "must put in an appearance there,\n",
      "the prince\n",
      "My daughter is\n",
      "coming for me to take me there.\n",
      "I thought today's fete had been canceled. I confess all these\n",
      "festivities and fireworks are becoming wearisome.\n",
      "If they had known that you wished it, the entertainment would\n",
      "have been put off,\n",
      "the prince\n",
      "Don't tease! Well, and what has been decided about Novosiltsev's\n",
      "dispatch? You know everything.\n",
      "What can one say about it?\n",
      "the prince\n",
      "What has been decided? They have decided that\n",
      "Buonaparte has burnt his boats, and I believe that we are ready to\n",
      "burn ours.\n",
      "Prince Vasili\n",
      "Anna Pavlovna\n",
      "Anna Pavlovna\n",
      "Oh, don't speak to me of Austria. Perhaps I don't understand\n",
      "things, but Austria never has wished, and does not wish, for war.\n",
      "She is betraying us! Russia alone must save Europe. Our gracious\n",
      "sovereign recognizes his high vocation and will be true to it. That is\n",
      "the one thing I have faith in! Our good and wonderful sovereign has to\n",
      "perform the noblest role on earth, and he is so virtuous and noble\n",
      "that God will not forsake him. He will fulfill his vocation and\n",
      "crush the hydra of revolution, which has become more terrible than\n",
      "ever in the person of this murderer and villain! We alone must\n",
      "avenge the blood of the just one.... Whom, I ask you, can we rely\n",
      "on?... England with her commercial spirit will not and cannot\n",
      "understand the Emperor Alexander's loftiness of soul. She has\n",
      "refused to evacuate Malta. She wanted to find, and still seeks, some\n",
      "secret motive in our actions. What answer did Novosiltsev get? None.\n",
      "The English have not understood and cannot understand the\n",
      "self-abnegation of our Emperor who wants nothing for himself, but only\n",
      "desires the good of mankind. And what have they promised? Nothing! And\n",
      "what little they have promised they will not perform! Prussia has\n",
      "always declared that Buonaparte is invincible, and that all Europe\n",
      "is powerless before him.... And I don't believe a word that Hardenburg\n",
      "says, or Haugwitz either. This famous Prussian neutrality is just a\n",
      "trap. I have faith only in God and the lofty destiny of our adored\n",
      "monarch. He will save Europe!\n",
      "I think,\n",
      "the prince\n",
      "that if you had been\n",
      "sent instead of our dear Wintzingerode you would have captured the\n",
      "King of Prussia's consent by assault. You are so eloquent. Will you\n",
      "give me a cup of tea?\n",
      "Wintzingerode\n",
      "King of Prussia\n",
      "In a moment. A propos,\n",
      "I am\n",
      "expecting two very interesting men tonight, le Vicomte de Mortemart,\n",
      "who is connected with the Montmorencys through the Rohans, one of\n",
      "the best French families. He is one of the genuine emigres, the good\n",
      "ones. And also the Abbe Morio. Do you know that profound thinker? He\n",
      "has been received by the Emperor. Had you heard?\n",
      "le Vicomte de Mortemart\n",
      "Montmorencys\n",
      "Rohans\n",
      "Abbe Morio\n",
      "the Emperor\n",
      "I shall be delighted to meet them,\n",
      "the prince\n",
      "But tell me,\n",
      "is it true that the Dowager Empress wants Baron Funke\n",
      "to be appointed first secretary at Vienna? The baron by all accounts\n",
      "is a poor creature.\n",
      "Prince Vasili\n",
      "Dowager Empress Marya Fedorovna\n",
      "the baron\n",
      "Anna Pavlovna\n",
      "the Empress\n",
      "Baron Funke has been recommended to the Dowager Empress by her\n",
      "sister,\n",
      "the Empress\n",
      "Anna Pavlovna's\n",
      "Her Majesty\n",
      "Baron\n",
      "Funke\n",
      "The prince\n",
      "Anna\n",
      "Pavlovna\n",
      "the Empress\n",
      "Now about your family. Do you know that since your daughter came\n",
      "out everyone has been enraptured by her? They say she is amazingly\n",
      "beautiful.\n",
      "The prince\n",
      "I often think,\n",
      "I often think how unfairly sometimes the\n",
      "joys of life are distributed. Why has fate given you two such splendid\n",
      "children? I don't speak of Anatole, your youngest. I don't like\n",
      "him,\n",
      "Anatole\n",
      "Two such charming children. And really you appreciate\n",
      "them less than anyone, and so you don't deserve to have them.\n",
      "I can't help it,\n",
      "the prince\n",
      "Lavater would have said I\n",
      "lack the bump of paternity.\n",
      "Don't joke; I mean to have a serious talk with you. Do you know I\n",
      "am dissatisfied with your younger son? Between ourselves\n",
      "he was mentioned at Her\n",
      "Majesty's and you were pitied....\n",
      "The prince\n",
      "What would you have me do?\n",
      "You know I did all\n",
      "a father could for their education, and they have both turned out\n",
      "fools. Hippolyte is at least a quiet fool, but Anatole is an active\n",
      "one. That is the only difference between them.\n",
      "And why are children born to such men as you? If you were not a\n",
      "father there would be nothing I could reproach you with,\n",
      "Anna\n",
      "Pavlovna\n",
      "I am your faithful slave and to you alone I can confess that my\n",
      "children are the bane of my life. It is the cross I have to bear. That\n",
      "is how I explain it to myself. It can't be helped!\n",
      "Anna Pavlovna\n"
     ]
    }
   ],
   "source": [
    "#.find_all('span', {'class':{'green', 'red'}})\n",
    "nameList = bs.findAll('span', {'class': ['green', 'red']})\n",
    "for name in nameList:\n",
    "    print(name.get_text())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Anna\n",
      "Pavlovna Scherer\n",
      "Empress Marya\n",
      "Fedorovna\n",
      "Prince Vasili Kuragin\n",
      "Anna Pavlovna\n",
      "St. Petersburg\n",
      "the prince\n",
      "Anna Pavlovna\n",
      "Anna Pavlovna\n",
      "the prince\n",
      "the prince\n",
      "the prince\n",
      "Prince Vasili\n",
      "Anna Pavlovna\n",
      "Anna Pavlovna\n",
      "the prince\n",
      "Wintzingerode\n",
      "King of Prussia\n",
      "le Vicomte de Mortemart\n",
      "Montmorencys\n",
      "Rohans\n",
      "Abbe Morio\n",
      "the Emperor\n",
      "the prince\n",
      "Prince Vasili\n",
      "Dowager Empress Marya Fedorovna\n",
      "the baron\n",
      "Anna Pavlovna\n",
      "the Empress\n",
      "the Empress\n",
      "Anna Pavlovna's\n",
      "Her Majesty\n",
      "Baron\n",
      "Funke\n",
      "The prince\n",
      "Anna\n",
      "Pavlovna\n",
      "the Empress\n",
      "The prince\n",
      "Anatole\n",
      "the prince\n",
      "The prince\n",
      "Anna\n",
      "Pavlovna\n",
      "Anna Pavlovna\n"
     ]
    }
   ],
   "source": [
    "nameList = bs.findAll('span', {'class': 'green'})\n",
    "for name in nameList:\n",
    "    print(name.get_text())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[<h1>War and Peace</h1>, <h2>Chapter 1</h2>]\n"
     ]
    }
   ],
   "source": [
    "titles = bs.find_all(['h1', 'h2','h3','h4','h5','h6'])\n",
    "print([title for title in titles])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[<span class=\"red\">Well, Prince, so Genoa and Lucca are now just family estates of the\n",
      "Buonapartes. But I warn you, if you don't tell me that this means war,\n",
      "if you still try to defend the infamies and horrors perpetrated by\n",
      "that Antichrist- I really believe he is Antichrist- I will have\n",
      "nothing more to do with you and you are no longer my friend, no longer\n",
      "my 'faithful slave,' as you call yourself! But how do you do? I see\n",
      "I have frightened you- sit down and tell me all the news.</span>, <span class=\"green\">Anna\n",
      "Pavlovna Scherer</span>, <span class=\"green\">Empress Marya\n",
      "Fedorovna</span>, <span class=\"green\">Prince Vasili Kuragin</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"green\">St. Petersburg</span>, <span class=\"red\">If you have nothing better to do, Count [or Prince], and if the\n",
      "prospect of spending an evening with a poor invalid is not too\n",
      "terrible, I shall be very charmed to see you tonight between 7 and 10-\n",
      "Annette Scherer.</span>, <span class=\"red\">Heavens! what a virulent attack!</span>, <span class=\"green\">the prince</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"red\">First of all, dear friend, tell me how you are. Set your friend's\n",
      "mind at rest,</span>, <span class=\"red\">Can one be well while suffering morally? Can one be calm in times\n",
      "like these if one has any feeling?</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"red\">You are\n",
      "staying the whole evening, I hope?</span>, <span class=\"red\">And the fete at the English ambassador's? Today is Wednesday. I\n",
      "must put in an appearance there,</span>, <span class=\"green\">the prince</span>, <span class=\"red\">My daughter is\n",
      "coming for me to take me there.</span>, <span class=\"red\">I thought today's fete had been canceled. I confess all these\n",
      "festivities and fireworks are becoming wearisome.</span>, <span class=\"red\">If they had known that you wished it, the entertainment would\n",
      "have been put off,</span>, <span class=\"green\">the prince</span>, <span class=\"red\">Don't tease! Well, and what has been decided about Novosiltsev's\n",
      "dispatch? You know everything.</span>, <span class=\"red\">What can one say about it?</span>, <span class=\"green\">the prince</span>, <span class=\"red\">What has been decided? They have decided that\n",
      "Buonaparte has burnt his boats, and I believe that we are ready to\n",
      "burn ours.</span>, <span class=\"green\">Prince Vasili</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"red\">Oh, don't speak to me of Austria. Perhaps I don't understand\n",
      "things, but Austria never has wished, and does not wish, for war.\n",
      "She is betraying us! Russia alone must save Europe. Our gracious\n",
      "sovereign recognizes his high vocation and will be true to it. That is\n",
      "the one thing I have faith in! Our good and wonderful sovereign has to\n",
      "perform the noblest role on earth, and he is so virtuous and noble\n",
      "that God will not forsake him. He will fulfill his vocation and\n",
      "crush the hydra of revolution, which has become more terrible than\n",
      "ever in the person of this murderer and villain! We alone must\n",
      "avenge the blood of the just one.... Whom, I ask you, can we rely\n",
      "on?... England with her commercial spirit will not and cannot\n",
      "understand the Emperor Alexander's loftiness of soul. She has\n",
      "refused to evacuate Malta. She wanted to find, and still seeks, some\n",
      "secret motive in our actions. What answer did Novosiltsev get? None.\n",
      "The English have not understood and cannot understand the\n",
      "self-abnegation of our Emperor who wants nothing for himself, but only\n",
      "desires the good of mankind. And what have they promised? Nothing! And\n",
      "what little they have promised they will not perform! Prussia has\n",
      "always declared that Buonaparte is invincible, and that all Europe\n",
      "is powerless before him.... And I don't believe a word that Hardenburg\n",
      "says, or Haugwitz either. This famous Prussian neutrality is just a\n",
      "trap. I have faith only in God and the lofty destiny of our adored\n",
      "monarch. He will save Europe!</span>, <span class=\"red\">I think,</span>, <span class=\"green\">the prince</span>, <span class=\"red\">that if you had been\n",
      "sent instead of our dear <span class=\"green\">Wintzingerode</span> you would have captured the\n",
      "<span class=\"green\">King of Prussia</span>'s consent by assault. You are so eloquent. Will you\n",
      "give me a cup of tea?</span>, <span class=\"green\">Wintzingerode</span>, <span class=\"green\">King of Prussia</span>, <span class=\"red\">In a moment. A propos,</span>, <span class=\"red\">I am\n",
      "expecting two very interesting men tonight, <span class=\"green\">le Vicomte de Mortemart</span>,\n",
      "who is connected with the <span class=\"green\">Montmorencys</span> through the <span class=\"green\">Rohans</span>, one of\n",
      "the best French families. He is one of the genuine emigres, the good\n",
      "ones. And also the <span class=\"green\">Abbe Morio</span>. Do you know that profound thinker? He\n",
      "has been received by <span class=\"green\">the Emperor</span>. Had you heard?</span>, <span class=\"green\">le Vicomte de Mortemart</span>, <span class=\"green\">Montmorencys</span>, <span class=\"green\">Rohans</span>, <span class=\"green\">Abbe Morio</span>, <span class=\"green\">the Emperor</span>, <span class=\"red\">I shall be delighted to meet them,</span>, <span class=\"green\">the prince</span>, <span class=\"red\">But tell me,</span>, <span class=\"red\">is it true that the Dowager Empress wants Baron Funke\n",
      "to be appointed first secretary at Vienna? The baron by all accounts\n",
      "is a poor creature.</span>, <span class=\"green\">Prince Vasili</span>, <span class=\"green\">Dowager Empress Marya Fedorovna</span>, <span class=\"green\">the baron</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"green\">the Empress</span>, <span class=\"red\">Baron Funke has been recommended to the Dowager Empress by her\n",
      "sister,</span>, <span class=\"green\">the Empress</span>, <span class=\"green\">Anna Pavlovna's</span>, <span class=\"green\">Her Majesty</span>, <span class=\"green\">Baron\n",
      "Funke</span>, <span class=\"green\">The prince</span>, <span class=\"green\">Anna\n",
      "Pavlovna</span>, <span class=\"green\">the Empress</span>, <span class=\"red\">Now about your family. Do you know that since your daughter came\n",
      "out everyone has been enraptured by her? They say she is amazingly\n",
      "beautiful.</span>, <span class=\"green\">The prince</span>, <span class=\"red\">I often think,</span>, <span class=\"red\">I often think how unfairly sometimes the\n",
      "joys of life are distributed. Why has fate given you two such splendid\n",
      "children? I don't speak of <span class=\"green\">Anatole</span>, your youngest. I don't like\n",
      "him,</span>, <span class=\"green\">Anatole</span>, <span class=\"red\">Two such charming children. And really you appreciate\n",
      "them less than anyone, and so you don't deserve to have them.</span>, <span class=\"red\">I can't help it,</span>, <span class=\"green\">the prince</span>, <span class=\"red\">Lavater would have said I\n",
      "lack the bump of paternity.</span>, <span class=\"red\">Don't joke; I mean to have a serious talk with you. Do you know I\n",
      "am dissatisfied with your younger son? Between ourselves</span>, <span class=\"red\">he was mentioned at Her\n",
      "Majesty's and you were pitied....</span>, <span class=\"green\">The prince</span>, <span class=\"red\">What would you have me do?</span>, <span class=\"red\">You know I did all\n",
      "a father could for their education, and they have both turned out\n",
      "fools. Hippolyte is at least a quiet fool, but Anatole is an active\n",
      "one. That is the only difference between them.</span>, <span class=\"red\">And why are children born to such men as you? If you were not a\n",
      "father there would be nothing I could reproach you with,</span>, <span class=\"green\">Anna\n",
      "Pavlovna</span>, <span class=\"red\">I am your faithful slave and to you alone I can confess that my\n",
      "children are the bane of my life. It is the cross I have to bear. That\n",
      "is how I explain it to myself. It can't be helped!</span>, <span class=\"green\">Anna Pavlovna</span>]\n"
     ]
    }
   ],
   "source": [
    "allText = bs.find_all('span', {'class':{'green', 'red'}})\n",
    "print([text for text in allText])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7\n"
     ]
    }
   ],
   "source": [
    "nameList = bs.find_all(text='the prince')\n",
    "print(len(nameList))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[<span class=\"red\">Well, Prince, so Genoa and Lucca are now just family estates of the\n",
      "Buonapartes. But I warn you, if you don't tell me that this means war,\n",
      "if you still try to defend the infamies and horrors perpetrated by\n",
      "that Antichrist- I really believe he is Antichrist- I will have\n",
      "nothing more to do with you and you are no longer my friend, no longer\n",
      "my 'faithful slave,' as you call yourself! But how do you do? I see\n",
      "I have frightened you- sit down and tell me all the news.</span>, <span class=\"green\">Anna\n",
      "Pavlovna Scherer</span>, <span class=\"green\">Empress Marya\n",
      "Fedorovna</span>, <span class=\"green\">Prince Vasili Kuragin</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"green\">St. Petersburg</span>, <span class=\"red\">If you have nothing better to do, Count [or Prince], and if the\n",
      "prospect of spending an evening with a poor invalid is not too\n",
      "terrible, I shall be very charmed to see you tonight between 7 and 10-\n",
      "Annette Scherer.</span>, <span class=\"red\">Heavens! what a virulent attack!</span>, <span class=\"green\">the prince</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"red\">First of all, dear friend, tell me how you are. Set your friend's\n",
      "mind at rest,</span>, <span class=\"red\">Can one be well while suffering morally? Can one be calm in times\n",
      "like these if one has any feeling?</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"red\">You are\n",
      "staying the whole evening, I hope?</span>, <span class=\"red\">And the fete at the English ambassador's? Today is Wednesday. I\n",
      "must put in an appearance there,</span>, <span class=\"green\">the prince</span>, <span class=\"red\">My daughter is\n",
      "coming for me to take me there.</span>, <span class=\"red\">I thought today's fete had been canceled. I confess all these\n",
      "festivities and fireworks are becoming wearisome.</span>, <span class=\"red\">If they had known that you wished it, the entertainment would\n",
      "have been put off,</span>, <span class=\"green\">the prince</span>, <span class=\"red\">Don't tease! Well, and what has been decided about Novosiltsev's\n",
      "dispatch? You know everything.</span>, <span class=\"red\">What can one say about it?</span>, <span class=\"green\">the prince</span>, <span class=\"red\">What has been decided? They have decided that\n",
      "Buonaparte has burnt his boats, and I believe that we are ready to\n",
      "burn ours.</span>, <span class=\"green\">Prince Vasili</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"red\">Oh, don't speak to me of Austria. Perhaps I don't understand\n",
      "things, but Austria never has wished, and does not wish, for war.\n",
      "She is betraying us! Russia alone must save Europe. Our gracious\n",
      "sovereign recognizes his high vocation and will be true to it. That is\n",
      "the one thing I have faith in! Our good and wonderful sovereign has to\n",
      "perform the noblest role on earth, and he is so virtuous and noble\n",
      "that God will not forsake him. He will fulfill his vocation and\n",
      "crush the hydra of revolution, which has become more terrible than\n",
      "ever in the person of this murderer and villain! We alone must\n",
      "avenge the blood of the just one.... Whom, I ask you, can we rely\n",
      "on?... England with her commercial spirit will not and cannot\n",
      "understand the Emperor Alexander's loftiness of soul. She has\n",
      "refused to evacuate Malta. She wanted to find, and still seeks, some\n",
      "secret motive in our actions. What answer did Novosiltsev get? None.\n",
      "The English have not understood and cannot understand the\n",
      "self-abnegation of our Emperor who wants nothing for himself, but only\n",
      "desires the good of mankind. And what have they promised? Nothing! And\n",
      "what little they have promised they will not perform! Prussia has\n",
      "always declared that Buonaparte is invincible, and that all Europe\n",
      "is powerless before him.... And I don't believe a word that Hardenburg\n",
      "says, or Haugwitz either. This famous Prussian neutrality is just a\n",
      "trap. I have faith only in God and the lofty destiny of our adored\n",
      "monarch. He will save Europe!</span>, <span class=\"red\">I think,</span>, <span class=\"green\">the prince</span>, <span class=\"red\">that if you had been\n",
      "sent instead of our dear <span class=\"green\">Wintzingerode</span> you would have captured the\n",
      "<span class=\"green\">King of Prussia</span>'s consent by assault. You are so eloquent. Will you\n",
      "give me a cup of tea?</span>, <span class=\"green\">Wintzingerode</span>, <span class=\"green\">King of Prussia</span>, <span class=\"red\">In a moment. A propos,</span>, <span class=\"red\">I am\n",
      "expecting two very interesting men tonight, <span class=\"green\">le Vicomte de Mortemart</span>,\n",
      "who is connected with the <span class=\"green\">Montmorencys</span> through the <span class=\"green\">Rohans</span>, one of\n",
      "the best French families. He is one of the genuine emigres, the good\n",
      "ones. And also the <span class=\"green\">Abbe Morio</span>. Do you know that profound thinker? He\n",
      "has been received by <span class=\"green\">the Emperor</span>. Had you heard?</span>, <span class=\"green\">le Vicomte de Mortemart</span>, <span class=\"green\">Montmorencys</span>, <span class=\"green\">Rohans</span>, <span class=\"green\">Abbe Morio</span>, <span class=\"green\">the Emperor</span>, <span class=\"red\">I shall be delighted to meet them,</span>, <span class=\"green\">the prince</span>, <span class=\"red\">But tell me,</span>, <span class=\"red\">is it true that the Dowager Empress wants Baron Funke\n",
      "to be appointed first secretary at Vienna? The baron by all accounts\n",
      "is a poor creature.</span>, <span class=\"green\">Prince Vasili</span>, <span class=\"green\">Dowager Empress Marya Fedorovna</span>, <span class=\"green\">the baron</span>, <span class=\"green\">Anna Pavlovna</span>, <span class=\"green\">the Empress</span>, <span class=\"red\">Baron Funke has been recommended to the Dowager Empress by her\n",
      "sister,</span>, <span class=\"green\">the Empress</span>, <span class=\"green\">Anna Pavlovna's</span>, <span class=\"green\">Her Majesty</span>, <span class=\"green\">Baron\n",
      "Funke</span>, <span class=\"green\">The prince</span>, <span class=\"green\">Anna\n",
      "Pavlovna</span>, <span class=\"green\">the Empress</span>, <span class=\"red\">Now about your family. Do you know that since your daughter came\n",
      "out everyone has been enraptured by her? They say she is amazingly\n",
      "beautiful.</span>, <span class=\"green\">The prince</span>, <span class=\"red\">I often think,</span>, <span class=\"red\">I often think how unfairly sometimes the\n",
      "joys of life are distributed. Why has fate given you two such splendid\n",
      "children? I don't speak of <span class=\"green\">Anatole</span>, your youngest. I don't like\n",
      "him,</span>, <span class=\"green\">Anatole</span>, <span class=\"red\">Two such charming children. And really you appreciate\n",
      "them less than anyone, and so you don't deserve to have them.</span>, <span class=\"red\">I can't help it,</span>, <span class=\"green\">the prince</span>, <span class=\"red\">Lavater would have said I\n",
      "lack the bump of paternity.</span>, <span class=\"red\">Don't joke; I mean to have a serious talk with you. Do you know I\n",
      "am dissatisfied with your younger son? Between ourselves</span>, <span class=\"red\">he was mentioned at Her\n",
      "Majesty's and you were pitied....</span>, <span class=\"green\">The prince</span>, <span class=\"red\">What would you have me do?</span>, <span class=\"red\">You know I did all\n",
      "a father could for their education, and they have both turned out\n",
      "fools. Hippolyte is at least a quiet fool, but Anatole is an active\n",
      "one. That is the only difference between them.</span>, <span class=\"red\">And why are children born to such men as you? If you were not a\n",
      "father there would be nothing I could reproach you with,</span>, <span class=\"green\">Anna\n",
      "Pavlovna</span>, <span class=\"red\">I am your faithful slave and to you alone I can confess that my\n",
      "children are the bane of my life. It is the cross I have to bear. That\n",
      "is how I explain it to myself. It can't be helped!</span>, <span class=\"green\">Anna Pavlovna</span>]\n"
     ]
    }
   ],
   "source": [
    "title = bs.find_all(id='title', class_='text')\n",
    "print([text for text in allText])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "<tr><th>\n",
      "Item Title\n",
      "</th><th>\n",
      "Description\n",
      "</th><th>\n",
      "Cost\n",
      "</th><th>\n",
      "Image\n",
      "</th></tr>\n",
      "\n",
      "\n",
      "<tr class=\"gift\" id=\"gift1\"><td>\n",
      "Vegetable Basket\n",
      "</td><td>\n",
      "This vegetable basket is the perfect gift for your health conscious (or overweight) friends!\n",
      "<span class=\"excitingNote\">Now with super-colorful bell peppers!</span>\n",
      "</td><td>\n",
      "$15.00\n",
      "</td><td>\n",
      "<img src=\"../img/gifts/img1.jpg\"/>\n",
      "</td></tr>\n",
      "\n",
      "\n",
      "<tr class=\"gift\" id=\"gift2\"><td>\n",
      "Russian Nesting Dolls\n",
      "</td><td>\n",
      "Hand-painted by trained monkeys, these exquisite dolls are priceless! And by \"priceless,\" we mean \"extremely expensive\"! <span class=\"excitingNote\">8 entire dolls per set! Octuple the presents!</span>\n",
      "</td><td>\n",
      "$10,000.52\n",
      "</td><td>\n",
      "<img src=\"../img/gifts/img2.jpg\"/>\n",
      "</td></tr>\n",
      "\n",
      "\n",
      "<tr class=\"gift\" id=\"gift3\"><td>\n",
      "Fish Painting\n",
      "</td><td>\n",
      "If something seems fishy about this painting, it's because it's a fish! <span class=\"excitingNote\">Also hand-painted by trained monkeys!</span>\n",
      "</td><td>\n",
      "$10,005.00\n",
      "</td><td>\n",
      "<img src=\"../img/gifts/img3.jpg\"/>\n",
      "</td></tr>\n",
      "\n",
      "\n",
      "<tr class=\"gift\" id=\"gift4\"><td>\n",
      "Dead Parrot\n",
      "</td><td>\n",
      "This is an ex-parrot! <span class=\"excitingNote\">Or maybe he's only resting?</span>\n",
      "</td><td>\n",
      "$0.50\n",
      "</td><td>\n",
      "<img src=\"../img/gifts/img4.jpg\"/>\n",
      "</td></tr>\n",
      "\n",
      "\n",
      "<tr class=\"gift\" id=\"gift5\"><td>\n",
      "Mystery Box\n",
      "</td><td>\n",
      "If you love suprises, this mystery box is for you! Do not place on light-colored surfaces. May cause oil staining. <span class=\"excitingNote\">Keep your friends guessing!</span>\n",
      "</td><td>\n",
      "$1.50\n",
      "</td><td>\n",
      "<img src=\"../img/gifts/img6.jpg\"/>\n",
      "</td></tr>\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from urllib.request import urlopen\n",
    "from bs4 import BeautifulSoup\n",
    "\n",
    "html = urlopen('http://www.pythonscraping.com/pages/page3.html')\n",
    "bs = BeautifulSoup(html, 'html.parser')\n",
    "\n",
    "for child in bs.find('table',{'id':'giftList'}).children:\n",
    "    print(child)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "<tr class=\"gift\" id=\"gift1\"><td>\n",
      "Vegetable Basket\n",
      "</td><td>\n",
      "This vegetable basket is the perfect gift for your health conscious (or overweight) friends!\n",
      "<span class=\"excitingNote\">Now with super-colorful bell peppers!</span>\n",
      "</td><td>\n",
      "$15.00\n",
      "</td><td>\n",
      "<img src=\"../img/gifts/img1.jpg\"/>\n",
      "</td></tr>\n",
      "\n",
      "\n",
      "<tr class=\"gift\" id=\"gift2\"><td>\n",
      "Russian Nesting Dolls\n",
      "</td><td>\n",
      "Hand-painted by trained monkeys, these exquisite dolls are priceless! And by \"priceless,\" we mean \"extremely expensive\"! <span class=\"excitingNote\">8 entire dolls per set! Octuple the presents!</span>\n",
      "</td><td>\n",
      "$10,000.52\n",
      "</td><td>\n",
      "<img src=\"../img/gifts/img2.jpg\"/>\n",
      "</td></tr>\n",
      "\n",
      "\n",
      "<tr class=\"gift\" id=\"gift3\"><td>\n",
      "Fish Painting\n",
      "</td><td>\n",
      "If something seems fishy about this painting, it's because it's a fish! <span class=\"excitingNote\">Also hand-painted by trained monkeys!</span>\n",
      "</td><td>\n",
      "$10,005.00\n",
      "</td><td>\n",
      "<img src=\"../img/gifts/img3.jpg\"/>\n",
      "</td></tr>\n",
      "\n",
      "\n",
      "<tr class=\"gift\" id=\"gift4\"><td>\n",
      "Dead Parrot\n",
      "</td><td>\n",
      "This is an ex-parrot! <span class=\"excitingNote\">Or maybe he's only resting?</span>\n",
      "</td><td>\n",
      "$0.50\n",
      "</td><td>\n",
      "<img src=\"../img/gifts/img4.jpg\"/>\n",
      "</td></tr>\n",
      "\n",
      "\n",
      "<tr class=\"gift\" id=\"gift5\"><td>\n",
      "Mystery Box\n",
      "</td><td>\n",
      "If you love suprises, this mystery box is for you! Do not place on light-colored surfaces. May cause oil staining. <span class=\"excitingNote\">Keep your friends guessing!</span>\n",
      "</td><td>\n",
      "$1.50\n",
      "</td><td>\n",
      "<img src=\"../img/gifts/img6.jpg\"/>\n",
      "</td></tr>\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from urllib.request import urlopen\n",
    "from bs4 import BeautifulSoup\n",
    "\n",
    "html = urlopen('http://www.pythonscraping.com/pages/page3.html')\n",
    "bs = BeautifulSoup(html, 'html.parser')\n",
    "\n",
    "for sibling in bs.find('table', {'id':'giftList'}).tr.next_siblings:\n",
    "    print(sibling) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "$15.00\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from urllib.request import urlopen\n",
    "from bs4 import BeautifulSoup\n",
    "\n",
    "html = urlopen('http://www.pythonscraping.com/pages/page3.html')\n",
    "bs = BeautifulSoup(html, 'html.parser')\n",
    "print(bs.find('img',\n",
    "              {'src':'../img/gifts/img1.jpg'})\n",
    "      .parent.previous_sibling.get_text())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "../img/gifts/img1.jpg\n",
      "../img/gifts/img2.jpg\n",
      "../img/gifts/img3.jpg\n",
      "../img/gifts/img4.jpg\n",
      "../img/gifts/img6.jpg\n"
     ]
    }
   ],
   "source": [
    "from urllib.request import urlopen\n",
    "from bs4 import BeautifulSoup\n",
    "import re\n",
    "\n",
    "html = urlopen('http://www.pythonscraping.com/pages/page3.html')\n",
    "bs = BeautifulSoup(html, 'html.parser')\n",
    "images = bs.find_all('img', {'src':re.compile('\\.\\.\\/img\\/gifts/img.*\\.jpg')})\n",
    "for image in images: \n",
    "    print(image['src'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<img src=\"../img/gifts/logo.jpg\" style=\"float:left;\"/>,\n",
       " <tr class=\"gift\" id=\"gift1\"><td>\n",
       " Vegetable Basket\n",
       " </td><td>\n",
       " This vegetable basket is the perfect gift for your health conscious (or overweight) friends!\n",
       " <span class=\"excitingNote\">Now with super-colorful bell peppers!</span>\n",
       " </td><td>\n",
       " $15.00\n",
       " </td><td>\n",
       " <img src=\"../img/gifts/img1.jpg\"/>\n",
       " </td></tr>,\n",
       " <tr class=\"gift\" id=\"gift2\"><td>\n",
       " Russian Nesting Dolls\n",
       " </td><td>\n",
       " Hand-painted by trained monkeys, these exquisite dolls are priceless! And by \"priceless,\" we mean \"extremely expensive\"! <span class=\"excitingNote\">8 entire dolls per set! Octuple the presents!</span>\n",
       " </td><td>\n",
       " $10,000.52\n",
       " </td><td>\n",
       " <img src=\"../img/gifts/img2.jpg\"/>\n",
       " </td></tr>,\n",
       " <tr class=\"gift\" id=\"gift3\"><td>\n",
       " Fish Painting\n",
       " </td><td>\n",
       " If something seems fishy about this painting, it's because it's a fish! <span class=\"excitingNote\">Also hand-painted by trained monkeys!</span>\n",
       " </td><td>\n",
       " $10,005.00\n",
       " </td><td>\n",
       " <img src=\"../img/gifts/img3.jpg\"/>\n",
       " </td></tr>,\n",
       " <tr class=\"gift\" id=\"gift4\"><td>\n",
       " Dead Parrot\n",
       " </td><td>\n",
       " This is an ex-parrot! <span class=\"excitingNote\">Or maybe he's only resting?</span>\n",
       " </td><td>\n",
       " $0.50\n",
       " </td><td>\n",
       " <img src=\"../img/gifts/img4.jpg\"/>\n",
       " </td></tr>,\n",
       " <tr class=\"gift\" id=\"gift5\"><td>\n",
       " Mystery Box\n",
       " </td><td>\n",
       " If you love suprises, this mystery box is for you! Do not place on light-colored surfaces. May cause oil staining. <span class=\"excitingNote\">Keep your friends guessing!</span>\n",
       " </td><td>\n",
       " $1.50\n",
       " </td><td>\n",
       " <img src=\"../img/gifts/img6.jpg\"/>\n",
       " </td></tr>]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bs.find_all(lambda tag: len(tag.attrs) == 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<span class=\"excitingNote\">Or maybe he's only resting?</span>]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bs.find_all(lambda tag: tag.get_text() == 'Or maybe he\\'s only resting?')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"Or maybe he's only resting?\"]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bs.find_all('', text='Or maybe he\\'s only resting?')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}