Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

an example would be great! #16

Open
seiyria opened this issue Feb 5, 2017 · 1 comment
Open

an example would be great! #16

seiyria opened this issue Feb 5, 2017 · 1 comment
Labels

Comments

@seiyria
Copy link

seiyria commented Feb 5, 2017

I'm trying to read the README and figure out based on my subroutines, evaluator, surgeon import, and html data how to mesh it all together.

it's tough to know how to go from the yaml config to something that actually works. for example, here is what I have so far:

import { ScraperOperation } from './_base';
import * as request from 'request';
import * as surgeon from 'surgeon';

const URL = 'http://mysite.com';

export class Scraper extends ScraperOperation {
  public scrape() {
    this.readYaml(`test.yaml`) //read as string
      .then(mainSiteInstructions => {
        console.log(mainSiteInstructions);
        request(URL, (err, res, body) => {
          if(err || res.statusCode !== 200) return;

          const x = surgeon.default({ evaluator: surgeon.cheerioEvaluator() });
          x(mainSiteInstructions, body);

        });
      });
  }
}

where the yaml file is:

alphabetLinks:
  - select a {0, } | rp title | test /Rule Index/

I just seem to continuously get errors though.

@seiyria
Copy link
Author

seiyria commented Feb 5, 2017

actually - I think this might work, my data source is just modifying itself because I don't have javascript enabled when doing a request(?) but all the same I think a full example would help a lot.

@gajus gajus added the question label Mar 15, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants