I have installed and configured the “Losant Edge Agent” on a Dell GW 3003. After adding the device, via the Cloud UI, and creating the
I tried to startup a docker instance of the Edge Agent. I noticed in the logs that we are get numerous “Illegal Instructions”. Can you let us know why this might be happening? One other caveat is that our Dell GW 3003 is also installed with a “Read Only” File System so we did not create any additional storage as specified in the “Run” command under “Configuration”. Is this required? What is the best way to debug this issue
Hi @Edward_Szajner,
I am going to look into this further, as nothing obvious is coming to mind that could be amiss. One of our engineers did inform me that the Dell GW 3003 is equipped Windows 10 IoT. I know that we have tested on Windows, but not specifically on Windows 10 IoT. I will keep looking into this and update you as soon as possible!
Thanks,
Julia
We are running Ubuntu on our Dell GW 3003 if that helps any. It appears to be a “Core” OS that also appears to be Read/Only as we are unable to create any directories. We are working with Dell to figure this out as well. It appears that these devices are equipped with some type of 8GB or 32GB eMMC internal storage. But it does not appear that not have presented the “volume” to the docker run command is the issue from what you are saying
Hi @Edward_Szajner,
I did some thinking about this. Overall everything should work without mounting the volume. Check out this doc on creating the storage area:
The one thing I’d recommend is to start your container with the LOGGER_LEVEL as verbose. If you could post an example of the logs you’re seeing, that would be super helpful. And, to verify, you see “Illegal Instructions” in the edge agent logs correct?
I am not sure changing the “LOGGER_LEVEL” of the container is going to help much in this situation as the “Losant Edge” container continuously reboots every 15 seconds or so. The logs contains only this line over and over again. Once for every restart
Illegal instruction
Not much help as this is a very terse message
thanks
ed
Here is some additional data
Here is the “Config” file that I used
C:\Users\v111405\Downloads>type config.iotgw
[logger]
out = "/data/losant-edge-agent-log.log’
level = ‘verbose’
[gateway]
id = ‘5d5ea5081a74840006eabffc’
key = ‘42530772-d9a7-44ca-8feb-fd87dXXXXXX’
secret = ‘6bf8896ce0502155959c3f92c040a12dc0becXXXXXXXXXXXX’
Here is the actual Docker command I used to start the “Losant Agent” container
C:\Users\v111405\Downloads>type dockerCommand
sudo docker run -d --restart always --name losant-agent -v /home/losant-edge-agent/data:/data -v /home/losant-edge-agent/config.iotgw:/etc/losant/losant-edge-agent-config.iotgw losant/edge-agent
ed
Hi @Edward_Szajner,
I apologize for the delay, this is not an issue we have run into before, so we have been discussing the possible causes. I received a few suggestions (and have a couple of questions as well) so we can get further into debugging! I apologize if any of these questions and suggestions are redundant to your own debugging 
- I am curious, is there a reason you are using a .iotgw file over a .toml file?
- Suggestion: Try removing the [logger] info (out and level as well) from toml (iotgw) file so it is not trying to write back to the read-only file system
- You said you were troubleshooting with Dell as well, were they able to provide any information?
- Suggestion: Try removing
-v /home/losant-edge-agent/data:/data from your docker run command so Docker is not trying to write back to the read-only file system
- You said earlier that you are running Ubuntu, but the commands look as though they may be for a Windows system. I just want to confirm that it is Ubuntu
Thank you so much for your patience and diligence in finding the cause of this! 
Julia
Julia,
Thanks for the update. Below you will find the answers to your questions
- I am curious, is there a reason you are using a .iotgw file over a .toml file?
[ECS] I was not under the impression that the .toml suffix was significant. I thought it might
be the name of the person preparing the video. I can certainly change that if you think
it plays a part in the issue.
- Suggestion: Try removing the [logger] info (out and level as well) from toml (iotgw) file so it is not trying to write back to the read-only file system
[ECS] The logging information was a suggestion from one of your colleagues, Taron Foxworth, as he
asked us to set “Logging Level” to “verbose”. Initially I did not have anything in there. Also, I
now have a “R/W” disk attached and this is where the log file should be getting written to. This
is also the volume that we are presenting to the container via the “-v” arguments in the docker
command. Initially we did not use a config file and only executed the CLI command to startup
the agent docker container with the same exact results. This implies that nothing in the config
file, contents, or .toml/.iotgw suffix are playing a role in this issue
- You said you were troubleshooting with Dell as well, were they able to provide any information?
[ECS] We only sent them an Email regarding the GW having a RO filesystem. They simply sent us
a user manual. We figured out on our own that we did in fact have a R/W partition in “/home”
which is where we create the “losant-edge-agent” and “losant-edge-agent/data” directories
that we present to the Losant edge agent docker container via the docker command. This
command is in a previous post for you review
- Suggestion: Try removing
-v /home/losant-edge-agent/data:/data from your docker run command so Docker is not trying to write back to the read-only file system
[ECS] This is not an issue as the directory “/home/losant-edge-agent/data” is RO for everyone. With
this permission set Docker should not have any issues writing to this directory
- You said earlier that you are running Ubuntu, but the commands look as though they may be for a Windows system. I just want to confirm that it is Ubuntu
[ECS] Here is the data that you requested
Looks like the data got truncated for the Linux (Ubuntu) verification. Here is another try
Directory command showing the permissions for /home/losant-edge-agent and /home/losant-edge-agent directories.
As you can
see the permissions are “rwxrwxrwx (777)” for both
total 16
drwxr-xr-x 4 root root 4096 Aug 27 13:49 …
-rw-rw-r-- 1 admin admin 231 Aug 29 15:14 config.iotgw
drwxrwxrwx 2 admin admin 4096 Aug 29 15:29 data
-rw-rw-r-- 1 admin admin 0 Aug 30 16:58 data.txt
drwxrwxrwx 3 admin admin 4096 Aug 30 16:58 .
Output from “uname -a” indicating a Ubuntu OS
Linux B7Z8N42 4.4.0-159-generic #187-Ubuntu SMP Thu Aug 1 16:28:06 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Hello Edward,
Sorry for the back and forth while we try and figure out the issue you are experiencing. With the Ubuntu CORE OS, I think you are installing Docker with snap. If so, can you uninstall Docker and try and install it again with this command:
sudo snap install docker --devmode
Once Docker is installed, you can follow the steps that you were already doing to pull the Losant Edge Agent Docker image and start the container. Here are the links.
You will have to change your config file extension from .iotgw to .toml as well. Just a reminder to do this at the file and also both places in the docker run command that it is referenced.
Let us know how it goes!
Thank you,
Kevin
1 Like
Unfortunately there is no change in behavior. I issued the following commands to remove the current docker image
sudo snap stop docker
sudo snap remove docker
Then I installed per your suggestion
sudo snap install docker --devmode
Then I pulled over the latest Losant Edge Agent
sudo docker pull losant/edge-agent
I also renamed my config file from config.iotgw to config.toml
Lastly I used the following command to start the Losant Edge Agent
sudo docker run -d --restart always --name losant-agent
-v /home/losant-edge-agent/data:/data
-v /home/losant-edge-agent/config.tom;:/etc/losant/losant-edge-agent-config.toml
losant/edge-agent
Not sure if I should start a different thread for this but I am having (what seems to be) the same issue.
I’m running an Advantech UTX-3115 with a fresh install of Ubuntu Server 18.04. Docker installed with apt, not snap, for the record.
Container appears to start, but when I check the logs I get “Illegal Instruction (Core Dumped)”
I have tried with and without a data directory, same issue.
Tried a fresh install of Ubuntu Server 18.04, and this time tried installing docker with
sudo snap install docker --devmode
as suggested above.
Followed install instruction exactly, config.toml file as follows:
[logger]
out = '/data/losant-edge-agent-log.log'
level = 'verbose'
[gateway]
id = '5d6cb82################'
key = 'ec15554e-1d57-4956-################'
secret = '4725780baceabab2a7a####################################'
[store]
path = '/data/losant-edge-agent-store.db'
docker run called as follows:
sudo docker run -d --restart always --name docs-agent -v /home/nated0g/losant/data:/data -v /home/nated0g/losant/config.toml:/etc/losant/losant-edge-agent-config.toml losant/edge-agent
Same issue, docker container reboots continuously, logs show only:
Illegal instruction (core dumped)
Illegal instruction (core dumped)
Illegal instruction (core dumped)
I tried an identical installation process on my Ubuntu 19.04 development machine and everything works properly and I can see the Edge Agent from Losant.
Not sure if relevant but I notice when I install docker, I get a warning:
aufs aufs_fill_super:912:mount[17219]: no arg
overlayfs: missing 'lowerdir'
@Edward_Szajner and @Nate_Usher,
I have been able to reproduce the Illegal instruction (core dumped) issue on our Edge Agent version 1.11.0. I tested this on an Advantech UTX-3115 gateway that is running Ubuntu 16.04 (I am not using snap). The issue appears to be limited to gateways that have an Atom processor.
I am working with Engineering on this issue now and will reply back as soon as I know more. Thank you for reporting this issue and your patience while we investigate.
Thank you Edward and Nate,
Kevin
1 Like
@Edward_Szajner and @Nate_Usher,
The Illegal Instruction (core dumped) error is limited to running the non-alpine version 1.10.0 or 1.11.0 of the Losant Edge Agent on an Atom device.
You can run the alpine version of the Losant Edge Agent. The alpine version does not come with the TensorFlow: Predict Node. The command to run the latest alpine version (1.11.0 at the time of this post) is sudo docker pull losant/edge-agent:1.11.0-alpine. You will have to change losant/edge-agent to losant/edge-agent:1.11.0-alpine at the end of your docker run command.
I will update this post when we have a fix for the non-alpine version.
Thank you,
Kevin
Kevin,
That worked except it did not like passing in the “configuration” (.toml) file. I had to use the following docker command
sudo docker run -d --restart always --name losant-agent -e ‘DEVICE_ID=5d5ea5081###############’ -e ‘ACCESS_KEY=42530772-d9a7-44ca-8feb-##############’ -e ‘ACCESS_SECRET=6bf8896ce0502155959c3f92c040a12d#########################’ losant/edge-agent:1.11.0-alpine
Here is the output from the docker logs command
admin@B7Z8N42:/home/losant-edge-agent$ sudo docker logs 17e3357953c0
2019-09-03T19:45:52.949Z [warn] This version of the edge agent does not support the TensorFlow Predict Node.
2019-09-03T19:45:54.626Z [info] Agent Starting…
2019-09-03T19:45:54.629Z [info] Agent Version: 1.11.0
2019-09-03T19:45:54.844Z [info] Webserver started on port: 8080
2019-09-03T19:45:54.853Z [info] Workflows initialized and running…
2019-09-03T19:45:54.856Z [info] Connecting to: mqtts://broker.losant.com …
2019-09-03T19:45:55.412Z [info] Connected to: mqtts://broker.losant.com
2019-09-03T19:45:55.637Z [info] Workflows received from Losant and applied.
Edward,
I am glad you got it working! Thank you for your patience as we worked through that issue.
In regards to the toml file issue: in your post from August 30th, the Docker run command that you were using was:
sudo docker run -d --restart always --name losant-agent -v /home/losant-edge-agent/data:/data -v /home/losant-edge-agent/config.tom;:/etc/losant/losant-edge-agent-config.toml losant/edge-agent
In the Docker run command above, the l in .toml is missing. You had a ; in place of the l which might have been a typo? When you executed that Docker run command last, was the ; still there instead of the l? Your updated Docker run command should be:
sudo docker run -d --restart always --name losant-agent -v /home/losant-edge-agent/data:/data -v /home/losant-edge-agent/config.toml:/etc/losant/losant-edge-agent-config.toml losant/edge-agent:1.11.0-alpine
Thank you Edward,
Kevin
Kevin,
I believe it was a typo. Here is the command I used from the “history”
sudo docker run -d --restart always --name losant-agent -v /home/losant-edge-agent/data:/data -v /home/losant-edge-agent/config.toml:/etc/losant/losant-edge-agent-config.toml losant/edge-agent:1.11.0-alpine
ed
Edward,
The next thing to check would be to ensure the config.toml file is really a toml file instead of something like config.toml.iotgw since you previously renamed your file extension from .iotgw to .toml. I have tried renaming a file extension before and it actually added what I thought was the new extension to the file name instead of updating the actual file extension.
If you decide to try and run with the toml file again, please post the error(s) that you receive.
Thank you,
Kevin
Kevin,
I did try again just to see what the error initially was. Here is the current log output
2019-09-03T20:34:13.034Z [warn] This version of the edge agent does not support the TensorFlow Predict Node.
2019-09-03T20:34:14.139Z [warn] Unable to read and parse agent config file at expected path: /etc/losant/losant-edge-agent-config.toml
2019-09-03T20:34:14.143Z [warn] Attempting startup using configuration from only environment variables and command line flags.
2019-09-03T20:34:14.168Z [error] Losant Edge Agent was unable to start due to the following configuration errors:
Device Id is required
2019-09-03T20:34:35.488Z [warn] This version of the edge agent does not support the TensorFlow Predict Node.
2019-09-03T20:34:36.617Z [warn] Unable to read and parse agent config file at expected path: /etc/losant/losant-edge-agent-config.toml
2019-09-03T20:34:36.619Z [warn] Attempting startup using configuration from only environment variables and command line flags.
2019-09-03T20:34:36.642Z [error] Losant Edge Agent was unable to start due to the following configuration errors:
Device Id is required
2019-09-03T20:34:57.580Z [warn] This version of the edge agent does not support the TensorFlow Predict Node.
2019-09-03T20:34:58.666Z [warn] Unable to read and parse agent config file at expected path: /etc/losant/losant-edge-agent-config.toml
2019-09-03T20:34:58.670Z [warn] Attempting startup using configuration from only environment variables and command line flags.
2019-09-03T20:34:58.694Z [error] Losant Edge Agent was unable to start due to the following configuration errors:
Device Id is required