Download Large File with HTTP Get Request

Hello,
I am trying to download a file from a server to Node Red then save as a file. The file is hosted on a server that responds to a GET request. Currently, the flow downloads part of the file using an HTTP request node and a file node, but not the whole thing. I can download the whole file with Postman, but using Node Red, i only get about 14kb of the 2.5MB file.

[{"id":"9d9d51eb.66525","type":"tab","label":"Download Pertus","disabled":false,"info":""},{"id":"741f50bf.487e7","type":"inject","z":"9d9d51eb.66525","name":"","topic":"DownloadPertu","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":310,"y":100,"wires":[["6aa4d239.1599ac"]]},{"id":"6aa4d239.1599ac","type":"change","z":"9d9d51eb.66525","name":"","rules":[{"t":"set","p":"timestamp","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":290,"y":140,"wires":[["26ecf671.880a6a"]]},{"id":"26ecf671.880a6a","type":"template","z":"9d9d51eb.66525","name":"Build msg.url","field":"url","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"http://172.1.1.1:80/peccgi/system/driver/HDMDRV?request=listFiles&group=PERTU","output":"str","x":270,"y":180,"wires":[["d2d5213d.67719"]]},{"id":"d2d5213d.67719","type":"http request","z":"9d9d51eb.66525","name":"","method":"GET","ret":"txt","paytoqs":false,"url":"","tls":"","persist":false,"proxy":"","authType":"","x":270,"y":220,"wires":[["7c00d646.1b6018"]]},{"id":"9f5148d7.327a18","type":"debug","z":"9d9d51eb.66525","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":250,"y":580,"wires":[]},{"id":"7c00d646.1b6018","type":"xml","z":"9d9d51eb.66525","name":"","property":"payload","attr":"","chr":"","x":250,"y":260,"wires":[["47b40763.a27678"]]},{"id":"47b40763.a27678","type":"change","z":"9d9d51eb.66525","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.hpc.StorageGroup.File","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":280,"y":300,"wires":[["f291e33.4d1972"]]},{"id":"363f2a70.80ad76","type":"http request","z":"9d9d51eb.66525","name":"","method":"GET","ret":"bin","paytoqs":false,"url":"","tls":"","persist":true,"proxy":"","authType":"","x":270,"y":460,"wires":[["3b0ba9ea.269ee6"]]},{"id":"3b0ba9ea.269ee6","type":"file","z":"9d9d51eb.66525","name":"","filename":"","appendNewline":false,"createDir":true,"overwriteFile":"true","encoding":"binary","x":250,"y":520,"wires":[["9f5148d7.327a18"]]},{"id":"f291e33.4d1972","type":"change","z":"9d9d51eb.66525","name":"","rules":[{"t":"set","p":"filename","pt":"msg","to":"payload[0].$.time","tot":"msg"},{"t":"set","p":"url","pt":"msg","to":"payload[0].$.url","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":280,"y":340,"wires":[["2a337b28.3eb6d4"]]},{"id":"2a337b28.3eb6d4","type":"function","z":"9d9d51eb.66525","name":"","func":"msg.filename = '/home/dev/unsecured/' + msg.filename + '.per';\nreturn msg;","outputs":1,"noerr":0,"x":250,"y":380,"wires":[["363f2a70.80ad76"]]}]

I’m not sure what could be preventing Node-RED or the HTTP request node in particular from getting the entire file, but could you try using an exec node with curl -X GET '<url>' – just to see if it has the same problem?

To the best of my knowledge, Node-RED should be able to handle up to 5MB files, so I am wondering whether or not there is a limitation on the HTTP request node itself.

So I tried with the curl request and got about 108kb instead of 14, so a little better. When i added --location and --request to the curl command, then it started working. Now I have it working with the HTTP node. I think i just had the url wrong?