[Solved] Endpoint GET -- JSON.parse(chunk) fails

Help
1

I am having trouble parsing information returned to my node.js application from an endpoint GET /devices request. I used JSON.parse(chunk) in the javascript code, where chunk is returned by the endpoint. The problem appears to be that too much data is being returned and the JSON.parse() call returns an error. It only happens when the chunk gets too big.

SyntaxError: Unexpected end of JSON input
at JSON.parse ()
at IncomingMessage.res.on (/home/pmmi/src/losantClient/losantClient.js:307:23)
at emitOne (events.js:96:13)
at IncomingMessage.emit (events.js:191:7)
at IncomingMessage.Readable.read (_stream_readable.js:383:10)
at flow (stream_readable.js:763:34)
at resume (_stream_readable.js:745:3)
at _combinedTickCallback (internal/process/next_tick.js:80:11)
at process._tickCallback (internal/process/next_tick.js:104:9)

Is there a way to parse the chunk without using JSON.parse()?

Ideally, the endpoint would return a simple array that looks something like this with each device representing one element of the array.

{[“name”: deviceName, “type”: deviceType, “id”: deviceId]}

This would dramatically cut down on the amount of data and perhaps avoid overloading JSON.parse().

Below is the debug output for the workflow.

{
“time”: “2017-05-08T21:29:04.962Z”,
“data”: {
“path”: “/devices”,
“params”: {},
“method”: “get”,
“headers”: {
“authorization”: “REDACTED”,
“content-type”: “application/json”,
“content-lenth”: “0”,
“connection”: “close”,
“x-forwarded-proto”: “http”,
“x-forwarded-for”: “107.182.103.167”
},
“query”: {},
“body”: null,
“cookies”: {},
“replyId”: “5908de3b11f27000011b2437.ryXRaMnk_.BytQmDC1ZE”,
“devices”: [
{
“name”: “Away”,
“tags”: {
“deviceType”: [
“Scene”
],
“owner”: [
“5903b1b43265360001666df3”
],
“manufacturerId”: [
“PMMI 00000001”
]
},
“attributes”: [],
“deviceClass”: “peripheral”,
“gatewayId”: “5910e311c2e38c0001dc7938”,
“applicationId”: “58f8fd024d33880001a45b0d”,
“creationDate”: “2017-05-08T21:28:59.840Z”,
“lastUpdated”: “2017-05-08T21:28:59.841Z”,
“deviceId”: “5910e31ba094a000012a7b36”,
“id”: “5910e31ba094a000012a7b36”
},
{
“name”: “Good Morning”,
“tags”: {
“deviceType”: [
“Scene”
],
“owner”: [
“5903b1b43265360001666df3”
],
“manufacturerId”: [
“PMMI 00000001”
]
},
“attributes”: [],
“deviceClass”: “peripheral”,
“gatewayId”: “5910e311c2e38c0001dc7938”,
“applicationId”: “58f8fd024d33880001a45b0d”,
“creationDate”: “2017-05-08T21:28:59.761Z”,
“lastUpdated”: “2017-05-08T21:28:59.762Z”,
“deviceId”: “5910e31b816fff0001b92439”,
“id”: “5910e31b816fff0001b92439”
},
{
“name”: “Good Night”,
“tags”: {
“deviceType”: [
“Scene”
],
“owner”: [
“5903b1b43265360001666df3”
],
“manufacturerId”: [
“PMMI 00000001”
]
},
“attributes”: [],
“deviceClass”: “peripheral”,
“gatewayId”: “5910e311c2e38c0001dc7938”,
“applicationId”: “58f8fd024d33880001a45b0d”,
“creationDate”: “2017-05-08T21:28:59.876Z”,
“lastUpdated”: “2017-05-08T21:28:59.876Z”,
“deviceId”: “5910e31ba094a000012a7b37”,
“id”: “5910e31ba094a000012a7b37”
},
{
“name”: “Home”,
“tags”: {
“deviceType”: [
“Scene”
],
“owner”: [
“5903b1b43265360001666df3”
],
“manufacturerId”: [
“PMMI 00000001”
]
},
“attributes”: [],
“deviceClass”: “peripheral”,
“gatewayId”: “5910e311c2e38c0001dc7938”,
“applicationId”: “58f8fd024d33880001a45b0d”,
“creationDate”: “2017-05-08T21:28:59.789Z”,
“lastUpdated”: “2017-05-08T21:28:59.789Z”,
“deviceId”: “5910e31b816fff0001b9243a”,
“id”: “5910e31b816fff0001b9243a”
},
{
“name”: “Snyder Testbed”,
“tags”: {
“deviceType”: [
“Controller”
],
“owner”: [
“5903b1b43265360001666df3”
],
“manufacturerId”: [
“PMMI 00000001”
]
},
“attributes”: [
{
“name”: “nAreas”,
“dataType”: “number”
},
{
“name”: “nScenes”,
“dataType”: “number”
}
],
“deviceClass”: “gateway”,
“applicationId”: “58f8fd024d33880001a45b0d”,
“creationDate”: “2017-05-08T21:28:49.933Z”,
“lastUpdated”: “2017-05-08T21:28:49.934Z”,
“deviceId”: “5910e311c2e38c0001dc7938”,
“id”: “5910e311c2e38c0001dc7938”
}
]
},
“applicationId”: “58f8fd024d33880001a45b0d”,
“triggerId”: “5908de3b11f27000011b2437”,
“triggerType”: “endpoint”,
“experience”: {
“endpoint”: {
“method”: “get”,
“enabled”: true,
“access”: “authenticated”,
“route”: “/devices”,
“applicationId”: “58f8fd024d33880001a45b0d”,
“creationDate”: “2017-05-02T19:30:03.009Z”,
“lastUpdated”: “2017-05-02T19:30:03.048Z”,
“experienceEndpointId”: “5908de3b11f27000011b2437”,
“id”: “5908de3b11f27000011b2437”,
“experienceGroups”: []
},
“user”: {
“email”: "my.awsome.user@example.com",
“userTags”: {},
“applicationId”: “58f8fd024d33880001a45b0d”,
“creationDate”: “2017-04-28T21:18:44.914Z”,
“lastUpdated”: “2017-05-01T18:30:09.915Z”,
“passwordLastUpdated”: “2017-04-28T21:18:44.915Z”,
“lastLogin”: “2017-05-01T18:30:09.906Z”,
“experienceUserId”: “5903b1b43265360001666df3”,
“avatarUrl”: “https://secure.gravatar.com/avatar/527b54db1085e9866e69a951690d3c3d?d=retro&s=240”,
“id”: “5903b1b43265360001666df3”,
“experienceGroups”: []
}
},
“relayId”: “5903b1b43265360001666df3”,
“relayType”: “experienceUser”,
“flowId”: “5908de9eb420670001553bd8”,
“flowName”: “GET /devices”,
“applicationName”: “PMMI Lighting”,
“globals”: {}
}

2

Any chance you can post the relevant code from your Node.js application as well?

3

Dylan,

I found the problem. The GET /devices request was returning multiple chunks asynchronously. I had to add code to my nodejs script to build up the chunk data in a local variable until all chunks were there before processing, otherwise the JSON.parse() call was being fed incomplete data. I did this by comparing the ‘content-length’ from the header with the sum of actual chunk sizes received until they added up.

Consider this closed. Thanks.

4

Glad to hear. Let us know if you have any more platform questions.