Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
    Howard Camp
    I was reading this guide to installing the Mercury API on AWS Lambda - https://www.evernote.com/shard/s3/client/snv?noteGuid=e8251e3d-3938-47bb-9941-64bb7c6f57f2&noteKey=2a89a5cbc811cfa4&sn=https%3A%2F%2Fwww.evernote.com%2Fshard%2Fs3%2Fsh%2Fe8251e3d-3938-47bb-9941-64bb7c6f57f2%2F2a89a5cbc811cfa4&title=Installing%2Bthe%2BMercury%2BReader%2BAPI%2Bon%2BAWS%2BLambda - Can anyone explain the AWS resources (aside from Lambda) that this uses? I notice for example a massive number of S3 requests associated with each call to the Mercury Parse API via AWS Lambda.
    Richard Fairbanks
    Greetings, folks!
    I have posted the solution to my Mac shell-script challenge with the Mercury Web Parser at:
    It has the AppleScript I’ve been using and the HTML template called in the script, as well as a screen shot of how it looks on my iPhone 7 Plus.
    It’s working great!
    Blessings, and thank you!!
    Hi, I am seeing the mercury-parser CLI is very slow processing the web pages, the CLI takes around 15s to process an URL, whereas the post light hosted web service used to take around 5s. Any thoughts on where the bottleneck could be. I am running the mercury-parser CLI on a Mac.
    Richard Fairbanks
    I can concur with tgima’s reported fifteen-second delay, running the mercury-parser CLI on a Mac.
    Howard Camp
    @adampash Are there instructions for hosting this within a simple Node app as opposed to on AWS Lambda?
    Steve Upstill
    @conradj , I forked your now-mercury parser, and got it deployed to Mow (as now-mercury-parser.upstill.now.sh), but when I hit it with a URL, it redirects to a URL that is identical except that it changes '://' to ':/'. I haven't touched any of the code. Does the parser work properly for you?
    Steve Upstill
    @MastaBaba My thanks as well for providing the solution that got me rolling. Server now running on my Linode!
    Babak Fakhamzadeh
    You're welcome @upstill :)
    Hey, im trying to install and run Mercury Web Parser but i seem to be running in to some problems
    Is someone here who could help me
    What's the best way to keep AWS updated with the latest Github changes?
    Do I have to repeat the yarn deploy steps @adampash laid out all over again or is there some automated way to handle this? I'm finding the documentation available across the internet to be not well-focused to say the least.
    Bryan Hackett
    Node 8.10 is losing support on AWS after 12/31. Are there any plans to update to support a newer version?
    @BryanHackett_twitter This Gitter space seems to have gone dark, which is a concern as I don't know where we are supposed to get information. Perhaps there is a space attached to the GitHub repository?

    Hi, could you help me please with passing errors and running .preview script? (PowerShell, Win10-64bit)


    node ./preview https://archiweb.cz/n/domaci/v-opave-se-bude-stavet-novy-bazen-za-350-milionu-korun                                      Rebuilding Mercury
    'MERCURY_TEST_BUILD' is not recognized as an internal or external command,
    operable program or batch file.
        throw err;
    Error: Command failed: MERCURY_TEST_BUILD=true npm run build
    'MERCURY_TEST_BUILD' is not recognized as an internal or external command,
    operable program or batch file.
        at checkExecSyncError (child_process.js:610:11)
        at execSync (child_process.js:646:15)
        at Object.<anonymous> (C:\app.martin\mercury-parser\mercury-parser\preview:20:3)
        at Module._compile (internal/modules/cjs/loader.js:1139:30)
        at Object.Module._extensions..js (internal/modules/cjs/loader.js:1159:10)
        at Module.load (internal/modules/cjs/loader.js:988:32)
        at Function.Module._load (internal/modules/cjs/loader.js:896:14)
        at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
        at internal/main/run_main_module.js:17:47 {
      status: 1,
      signal: null,
      output: [
        <Buffer >,
        <Buffer 27 4d 45 52 43 55 52 59 5f 54 45 53 54 5f 42 55 49 4c 44 27 20 69 73 20 6e 6f 74 20 72 65 63 6f 67 6e 69 7a 65 64 20 61 73 20 61 6e 20 69 6e 74 65 72 ... 59 more bytes>
      pid: 26728,
      stdout: <Buffer >,
      stderr: <Buffer 27 4d 45 52 43 55 52 59 5f 54 45 53 54 5f 42 55 49 4c 44 27 20 69 73 20 6e 6f 74 20 72 65 63 6f 67 6e 69 7a 65 64 20 61 73 20 61 6e 20 69 6e 74 65 72 ... 59 more bytes>
    Adam Pash
    @singularita-zz I don't have a windows machine to test on but i'm guessing that declaring the environment variable in the command MERCURY_TEST_BUILD=true npm run build isn't supported on powershell? you may have to edit the preview script to play friendly with powershell. it assumes a *nix shell
    @zootooz sorry for missing this: like you suggested, you would have to re-deploy
    @BryanHackett_twitter Apologies for the slow response. A couple of weeks ago, we updated the parser api to a newer node :thumbsup:
    Hi, how make custom extractor with Mercury API on AWS Lambda?
    @zootooz Ok, how can I then transfer this to my lambda? I use: https://github.com/postlight/mercury-parser-api
    I'm no expert here, but assuming you've already set up your lambda/mercury aws server, I believe you just have to re-deploy the files up to AWS.
    So for me that would be yarn deploy:prod
    @zootooz thanks you
    Dan Taylor
    Hey all, any advice on how to create a customExtractor to get the date on this page? https://www.90min.com/posts/afc-bournemouth-must-replace-key-stars-for-serious-promotion-push
    Matthew Krieger
    I get an enormous amount of http 502 bad gateway responses - is a 502 bad gateway generated by the parser API when it fails to download and or extract the web page, or is the 502 just passed from an upstream server to the web parser api which then returns it to me? In my individual testing I never have an issue browsing directly to the URLs that Mercury tells me 502 bad gateway for, but when they go through Mercury I get that error.
    Thomas Ladd
    Is it possible to add a generic custom extractor? I want to pull out some additional info for every domain that I parse. From perusing the docs and source code I think the answer is no.