Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Out of memory error when passing in sentence with no punctuation #5

Open
ghost opened this issue Jan 12, 2018 · 0 comments
Open

Out of memory error when passing in sentence with no punctuation #5

ghost opened this issue Jan 12, 2018 · 0 comments
Assignees

Comments

@ghost
Copy link

ghost commented Jan 12, 2018

I'm having issues attempting to pass in non-punctuated data. Passing in the below string kills javascript with an JavaScript heap out of memory error unless a period is appended to the output.

const generator = require('ngram-natural-language-generator').generator;

var text = ''
  + 'Lorem ipsum dolor sit amet consectetur adipiscing elit '
  + 'Suspendisse sit amet neque ipsum Sed sed lorem eu dolor '
  + 'sagittis aliquet sed sollicitudin ligula Nullam tempus elit'

generator({
  text: text + '.',
  model: {
    maxLength: 100,
    minLength: 10
  }
}, function(err, result) {
  console.log(result);
})


generator({
  text: text,
  model: {
    maxLength: 100,
    minLength: 10
  }
}, function(err, result) {
  console.log(result);
})

Output:

it Suspendisse sit amet consectetur adipiscing elit Suspendisse sit amet neque ipsum dolor sagittis aliquet sed

sollicitudin ligula Nullam tempus elit. Lorem ipsum Sed sed lorem eu dolor sit amet consectetur adipiscing elit
Suspendisse sit amet consectetur adipiscing elit. Lorem ipsum Sed sed lorem eu dolor sagittis aliquet sed sollic
itudin ligula Nullam tempus elit Suspendisse sit amet neque ipsum Sed sed sollicitudin ligula Nullam tempus elit
. Lorem ipsum Sed sed lorem eu dolor sit amet consectetur adipiscing elit. Lorem ipsum Sed sed sollicitudin ligu
la Nullam tempus elit Suspendisse sit amet neque ipsum dolor sagittis aliquet sed lorem eu dolor sagittis alique
t sed sollicitudin ligula Nullam tempus elit Suspendisse sit amet consectetur adipiscing elit Suspendisse sit am
et neque ipsum dolor sit amet consectetur adipiscing elit Suspendisse sit amet neque ipsum Sed sed lorem eu dolo
r sagittis aliquet sed lorem eu dolor sagittis aliquet sed lorem eu dolor sagittis aliquet sed sollicitudin ligu
la Nullam tempus elit Suspendisse sit amet neque ipsum Sed sed sollicitudin ligula Nullam tempus elit.

<--- Last few GCs --->

[6628:0000006D1FE253D0]     7040 ms: Mark-sweep 597.2 (604.4) -> 597.2 (604.4) MB, 281.4 / 0.0 ms  allocation fa
ilure GC in old space requested
[6628:0000006D1FE253D0]     7321 ms: Mark-sweep 597.2 (604.4) -> 597.2 (601.4) MB, 280.7 / 0.0 ms  last resort G
C in old space requested
[6628:0000006D1FE253D0]     7605 ms: Mark-sweep 597.2 (601.4) -> 597.2 (601.4) MB, 283.9 / 0.0 ms  last resort G
C in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000000978E7A5EC1 <JSObject>
    1: nlg(aka nlg) [C:\Users\UniQMG\Documents\Projects\node\Discord bots\markovic\node_modules\ngram-natural-la
nguage-generator\lib\nlg.js:~25] [pc=000001413B187220](this=00000008BBD82311 <undefined>,model=000003039AD692F1
<Object map = 0000014F60C45801>)
    3: /* anonymous */ [C:\Users\UniQMG\Documents\Projects\node\Discord bots\markovic\node_modules\ngram-natural
-language-generator\index.j...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Additionally it seems passing in very short strings, even without punctuation, throws an error w/o returning it to the callback passed:

const generator = require('ngram-natural-language-generator').generator;

generator({
  text: 'Hello world',
  model: {
    maxLength: 100,
    minLength: 10
  }
}, function(err, result) {
  console.log(result);
})
C:\Users\UniQMG\Documents\Projects\node\Discord bots\markovic\node_modules\ngram-natural-language-generator\lib\
nlg.js:12
    if (!words) throw new Error('Shouldnt be empty');
                ^

Error: Shouldnt be empty
    at choose (C:\Users\UniQMG\Documents\Projects\node\Discord bots\markovic\node_modules\ngram-natural-language
-generator\lib\nlg.js:12:23)

    at nlg (C:\Users\UniQMG\Documents\Projects\node\Discord bots\markovic\node_modules\ngram-natural-language-ge
nerator\lib\nlg.js:47:14)
    at C:\Users\UniQMG\Documents\Projects\node\Discord bots\markovic\node_modules\ngram-natural-language-generat
or\index.js:10:20
    at ngrams (C:\Users\UniQMG\Documents\Projects\node\Discord bots\markovic\node_modules\ngram-natural-language
-generator\lib\ngrams.js:98:9)
    at generator (C:\Users\UniQMG\Documents\Projects\node\Discord bots\markovic\node_modules\ngram-natural-langu
age-generator\index.js:7:3)
    at Object.<anonymous> (C:\Users\UniQMG\Documents\Projects\node\Discord bots\markovic\test.js:3:1)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
@cesine cesine self-assigned this Jan 18, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant