-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP: Support appendSeedData argument to be an array or array of arrays #18
Open
andreas-marschke
wants to merge
1
commit into
master
Choose a base branch
from
feature-csv-array
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
"use strict"; | ||
|
||
var csvStringify = require("csv-stringify"); | ||
|
||
/** | ||
* @module CSV | ||
* @memberof Util | ||
* @desc | ||
* Utility funcitons handling CSV data and Arrays | ||
*/ | ||
|
||
/** | ||
* @function isArray | ||
* | ||
* @desc | ||
* Test if the passed in argument is an array or not | ||
* | ||
* @param {} object - Argument tested for being an Array or not | ||
* | ||
* @returns {boolean} - true if the argument to the function is an Array | ||
*/ | ||
exports.isArray = function(object) { | ||
return Object.prototype.toString.call(object) === "[object Array]"; | ||
}; | ||
|
||
/** | ||
* @namespace csv | ||
* @function is2D | ||
* | ||
* @desc | ||
* Test if the elements in an already identified array are also an arrays | ||
* | ||
* _NOTE:_ This will also fail if only one element is also an array but the rest are singular values! | ||
* | ||
* @param {Object[]} array - An array to test for being a two dimensional array | ||
* | ||
* @returns {boolean} - true if the array is two dimensional | ||
*/ | ||
exports.is2D = function(array) { | ||
return array.filter(function(element) { return !exports.isArray(element) }).length === 0; | ||
}; | ||
|
||
/** | ||
* @namespace csv | ||
* @function toCSV | ||
* | ||
* @desc | ||
* Turn content into CSV formatted string | ||
* Uses {@link csv~is2D} and {@link csv~isArray} to test if the first argument is in the proper format | ||
* otherwise will simply return the content of the first argument, trusting that it is CSV | ||
* | ||
* @param {} content - Any type of object or array that can be turned into a 2D array for processing | ||
* @param {function} callback - | ||
* [Called once stringify has happend or it was determined not to be an array]{@link csv~toCSV(callback)} | ||
*/ | ||
exports.toCSV = function(content, callback) { | ||
if(exports.isArray(content)) { | ||
if (!exports.is2D(content)) { | ||
content = [content]; | ||
} | ||
csvStringify(content, function(err, csv) { | ||
callback(err, csv); | ||
}); | ||
} else { | ||
callback(null, content); | ||
} | ||
} | ||
/** | ||
* @callback csv~toCSV(callback) callback | ||
* | ||
* @param {?Error} error - {@link null} or an Error object describing the error that occured | ||
* @param {?string} csvString - {@link null} in Error case or CSV formatted string of data | ||
*/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a case where
api-json.js
is being used for a request that doesn't get back JSON?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. CSV data for example when we appendSeedData or readSeedData
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, I think that's a bug in
SeedData.readSeedData()
. It should be usingapiStream
instead ofapiJson
.There's no concern for
SeedData.appendSeedData()
, because the response is always empty (204 No Content).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well okay so we are wrapping appendSeedData into the stream and take
end
and the wrapper for the callback. That's just fine. Remember that you have 2 versions of append/read one stream one json. And one possibly doesn't work currently as it uses theapi-json.js
instead ofapi-stream.js
.Additionally using JSON.parse() and hoping to always retrieve JSON feels presumptious. What if the URL we are requesting this from is not an API endpoint, keep in mind the user may point it to another server not from us. What if the Server returns HTML because an Error happend internally and the Servlet was inaccessible? We should guard against these issues better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But then readSeedData and ReadSeedDataStream are redundant?
Am 22.09.2015 07:35 schrieb "Matt Solnit" [email protected]:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like switching to a single
readSeedData
that just reads a stream, instead of expecting it in JSON.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, so I have a few thoughts on this.
SeedData.readSeedData
definitely has a bug, and just as you found, if the response has multiple lines (which I never tested), you'll getSyntaxError: Unexpected string
. This is bad.Stream.Writable
, is more "pure" but inconvenient IMO.api-json.js
is also bad. We should letJSON.parse()
do its thing without having to worry about "expected non-JSON" responses.I propose that we keep
SeedData.readSeedData
, but instead of usingapiJson()
, it instead creates a temporary stream, callsapiStream()
, and then "un-wraps" the content, returning it to the caller as a regular string. In fact, it can even usethis.readSeedDataStream()
with an in-between callback to do the un-wrapping.Finally, we can add a note to the JSDoc, warning against doing this unless you know the response is small.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds fair.