Java, Ubuntu and reading serial ports

Discussion in 'Computer Science & Culture' started by AlphaNumeric, Sep 30, 2012.

Not open for further replies.
1. AlphaNumericFully ionizedRegistered Senior Member

Messages:
6,702
I have 64bit Ubuntu 12.04, fresh install. I want to do this except every guide seems to fail to work. The standard package is javax.comm or gnu.io.* . The latter (possibly both) involves installing a rxtx library. I have the library in an archive but part of the installation step is putting the right files into my jre/.... folder, except I don't know which one because I have anywhere from 4 to 10 different directories within my /usr/lib/jvm/ directory, depending on which java packages I install using Synaptic (java-7-openjdk-amd64, java-6-openjdk-amd64,java-1.5.0-gcj-4.6,java-7-oracle, etc. Installed Netbeans makes it go from 2 to 8!!). I have a test java file which is to be run via 'javac TestFile.java' , such as the one here. The problem is that no matter what I've installed or which guide I've followed it immediately fails by complaining it can't find the rxtx library, be it the gnu.io.* or javax.comm. According to Synaptic I have version 2.2 of the rxtx.com library installed. I've tried installing Java by installing Netbeans or Eclipse or both, adding libraries, pointing to files, moving .jar files about. Nothing allows me to run ANY file which involves the com port libraries! I must have wasted 5 hours on it today.

To give some context, I'm messing around with an Arduino Uno. I have installed the Arduino software via the Ubuntu manager and I can run it, upload programs and even use it to monitor the serial connection, correctly displaying the information I expect. But when I try to get Java to read the serial connection so I can do more elaborate things with the data, on a computer with a processor faster than 16MHz(!), and then send stuff back to the Arduino but I can't get Java to even build the program!! I have a book which is all about Arduino projects and it has a step by step guide to doing this in Python, which works, but I want to do it in Java due to the stuff I want to do with the data being already in Java. There's an Arduino/Eclipse IDE but Eclipse instantly crashes in 64bit Ubuntu 12.04. Fan-****ing-tastic.

What I want is just to have a working connection and a simple java file which reads the serial connection and, for instance, prints it to screen. Once I've got that I can move on to more complicated things, but I can't even install the rxtx library for Java! Please, if anyone knows the sequence of commands to set up Ubuntu so this file or even something which is just 'import javax.comm' works when you do 'javac File.java' I would be extremely grateful. I'm currently a Java novice and part of the reason for doing this is to have a fun way of getting into Java. Currently all it makes me want to do is punch the person who wrote the rxtx library for making it so bloody convoluted to install!

3. ChipzBannedBanned

Messages:
838
First, welcome to the disgusting world of java virtual machine versioning.

Like most uncommon packages, it's typically best to not use the package manager unless you have taken precautions. I successfully compiled this example http://rxtx.qbang.org/wiki/index.php/Two_way_communcation_with_the_serial_port , however I don't have a serial device to test with.

Let's first build the RXTX library.
Code:
$> cd /var/tmp$> wget http://rxtx.qbang.org/pub/rxtx/rxtx-2.1-7r2.zip
$> unzip rxtx-2.1-7r2.zip$> cd rxtx-2.1.7r2

Now we need to define where it should look for Java.

Code:
$> java -version java version "1.6.0_24" OpenJDK Runtime Environment (IcedTea6 1.11.1) (fedora-65.1.11.1.fc16-x86_64) OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)  Note: There's a reasonable chance your system is trying to default to GCJ vs openJDK, if this is the case remove remnants of your old GCJ compiler with an uninstall. The GNU std java is not compatible with many cross platform libraries or will behave in unexpected ways. Open another terminal to verify some things are set up. Code: $> cd /var/lib/jvm
$> ls -l lrwxrwxrwx. 1 root root 26 Mar 18 2012 java -> /etc/alternatives/java_sdk drwxr-xr-x. 3 root root 4096 May 10 20:21 java-1.5.0-gcj-1.5.0.0 lrwxrwxrwx. 1 root root 32 Mar 18 2012 java-1.6.0 -> /etc/alternatives/java_sdk_1.6.0 drwxr-xr-x. 7 root root 4096 Feb 15 2012 java-1.6.0-openjdk-1.6.0.0.x86_64 lrwxrwxrwx. 1 root root 33 Mar 18 2012 java-1.6.0-openjdk.x86_64 -> java-1.6.0-openjdk-1.6.0.0.x86_64 lrwxrwxrwx. 1 root root 26 Jan 3 2012 java-6-openjdk -> java-1.6.0-openjdk.x86_64/ lrwxrwxrwx. 1 root root 34 Mar 18 2012 java-openjdk -> /etc/alternatives/java_sdk_openjdk lrwxrwxrwx. 1 root root 21 May 10 20:21 jre -> /etc/alternatives/jre lrwxrwxrwx. 1 root root 27 May 10 20:21 jre-1.5.0 -> /etc/alternatives/jre_1.5.0 lrwxrwxrwx. 1 root root 26 May 10 20:21 jre-1.5.0-gcj -> java-1.5.0-gcj-1.5.0.0/jre lrwxrwxrwx. 1 root root 27 Mar 18 2012 jre-1.6.0 -> /etc/alternatives/jre_1.6.0 lrwxrwxrwx. 1 root root 37 Mar 18 2012 jre-1.6.0-openjdk.x86_64 -> java-1.6.0-openjdk-1.6.0.0.x86_64/jre lrwxrwxrwx. 1 root root 25 May 10 20:21 jre-gcj -> /etc/alternatives/jre_gcj lrwxrwxrwx. 1 root root 29 Mar 18 2012 jre-openjdk -> /etc/alternatives/jre_openjdk  In my case, the java version found was 1.6.0_24, meaning this is the directory I will be working with. In my case it's absolute path to /var/lib/jvm/java-1.6.0-openjdk.x86_64 which I will be working in. So we want to set the JAVA_HOME variable. In the same terminal that you had downloaded the rxtx source code execute: (Please substitute your version and path of Java on the RHS of the export) Code: $> export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk.x86_64

Now let's build and install.
Code:
$> ./configure ; make -j8  Now unfortunately the Makefile installer was wrong for me, so I have to manually install the files. First we need to move the JAR file. Code: $> sudo mv RXTXcomm.jar  /usr/lib/jvm/java-1.6.0-openjdk.x86_64/jre/lib/ext

Now, we need to move the shared libraries. For me it builds a folder called x86_64-unknown-linux-gnu, this is likely the same case for you, as you stated you are using X84_64

Code:
$> pwd /var/tmp/rxtx-2.1-7r2$> ls
....snip....
-rw-r--r--. 1 cjhanks cjhanks   9706 Apr  5  2002 TODO
drwxr-xr-x. 3 cjhanks cjhanks   4096 Jan 29  2006 WinCE
drwxrwxr-x. 3 cjhanks cjhanks   4096 Sep 30 14:51 x86_64-unknown-linux-gnu
$> cd x86_64-unknown-linux-gnu$> ls -la
drwxrwxr-x.  3 cjhanks cjhanks  4096 Sep 30 14:51 .
drwxr-xr-x. 11 cjhanks cjhanks  4096 Sep 30 14:51 ..
...snip...
drwxrwxr-x.  2 cjhanks cjhanks  4096 Sep 30 14:51 .libs
-rw-rw-r--.  1 cjhanks cjhanks   360 Sep 30 14:51 ParallelImp.lo
-rw-rw-r--.  1 cjhanks cjhanks   270 Sep 30 14:51 RawImp.loT
-rw-rw-r--.  1 cjhanks cjhanks   272 Sep 30 14:51 RS485Imp.loT
-rw-rw-r--.  1 cjhanks cjhanks   356 Sep 30 14:51 SerialImp.lo

$> cd .libs$> ls -l
-rw-rw-r--. 1 cjhanks cjhanks  55200 Sep 30 14:51 fuserImp.o
-rwxrwxr-x. 1 cjhanks cjhanks   5573 Sep 30 14:51 librxtxI2C-2.1-7.so
lrwxrwxrwx. 1 cjhanks cjhanks     16 Sep 30 14:51 librxtxI2C.la -> ../librxtxI2C.la
-rw-rw-r--. 1 cjhanks cjhanks    861 Sep 30 14:51 librxtxI2C.lai
lrwxrwxrwx. 1 cjhanks cjhanks     19 Sep 30 14:51 librxtxI2C.so -> librxtxI2C-2.1-7.so
-rwxrwxr-x. 1 cjhanks cjhanks   5573 Sep 30 14:51 librxtxParallel-2.1-7.so
lrwxrwxrwx. 1 cjhanks cjhanks     21 Sep 30 14:51 librxtxParallel.la -> ../librxtxParallel.la
-rw-rw-r--. 1 cjhanks cjhanks    896 Sep 30 14:51 librxtxParallel.lai
lrwxrwxrwx. 1 cjhanks cjhanks     24 Sep 30 14:51 librxtxParallel.so -> librxtxParallel-2.1-7.so
-rwxrwxr-x. 1 cjhanks cjhanks   5573 Sep 30 14:51 librxtxRaw-2.1-7.so
lrwxrwxrwx. 1 cjhanks cjhanks     16 Sep 30 14:51 librxtxRaw.la -> ../librxtxRaw.la
-rw-rw-r--. 1 cjhanks cjhanks    861 Sep 30 14:51 librxtxRaw.lai
lrwxrwxrwx. 1 cjhanks cjhanks     19 Sep 30 14:51 librxtxRaw.so -> librxtxRaw-2.1-7.so
-rwxrwxr-x. 1 cjhanks cjhanks   5573 Sep 30 14:51 librxtxRS485-2.1-7.so
lrwxrwxrwx. 1 cjhanks cjhanks     18 Sep 30 14:51 librxtxRS485.la -> ../librxtxRS485.la
-rw-rw-r--. 1 cjhanks cjhanks    875 Sep 30 14:51 librxtxRS485.lai
lrwxrwxrwx. 1 cjhanks cjhanks     21 Sep 30 14:51 librxtxRS485.so -> librxtxRS485-2.1-7.so
-rwxrwxr-x. 1 cjhanks cjhanks  42942 Sep 30 14:51 librxtxSerial-2.1-7.so
lrwxrwxrwx. 1 cjhanks cjhanks     19 Sep 30 14:51 librxtxSerial.la -> ../librxtxSerial.la
-rw-rw-r--. 1 cjhanks cjhanks    882 Sep 30 14:51 librxtxSerial.lai
lrwxrwxrwx. 1 cjhanks cjhanks     22 Sep 30 14:51 librxtxSerial.so -> librxtxSerial-2.1-7.so
-rw-rw-r--. 1 cjhanks cjhanks  69264 Sep 30 14:51 ParallelImp.o
-rw-rw-r--. 1 cjhanks cjhanks 243696 Sep 30 14:51 SerialImp.o

These are the compiled shared libraries, now we want to move these into the JRE folder. Please again take note that you replace the proper JRE path in my example.

Code:
$> sudo mv *.{so,la} /usr/lib/jvm/java-1.6.0-openjdk.x86_64/jre/lib/amd64  Now you should be done with installation. I copied this example: http://rxtx.qbang.org/wiki/index.php/Two_way_communcation_with_the_serial_port and saved it into a file called "TwoWaySerialComm.java". Code: $> javac TwoWaySerialComm.java

And it compiled with no errors, so it must have found the proper RXTX.jar file.

Now...
Your goal was to compile this program --> http://www.java-samples.com/showtutorial.php?tutorialid=11
However, that's using the library javax, which is not RXTX. Therefore this program will never compile without installing the javax library, which you don't need since you now have RXTX. Afait, RXTX is an updated port of javax rather than a co-dependency.

Best of luck.

Note:
It just occurred to me that you fresh install may not have some of the dependencies required to execute these steps. You may get some compilation errors which I can't recreate since I have 3 years of dependencies installed on my system. At the very least you will need
Code:
\$> sudo apt-get install build-essential


5. AlphaNumericFully ionizedRegistered Senior Member

Messages:
6,702
Thanks Chipz. I'll have a go through your guide when I get home this evening. Java communicating with an external device via USB seems like something so many people, particularly people building handheld devices (including phones!), would want that I'm amazed its so convoluted and the explanations so bad. The Arduino also has an ethernet 'shield' (extension board) which allows UDP packet communication. There's also a bluetooth extension and a wireless extension. I might try the ethernet one in the future, assuming I don't smash the Arduino and laptop into a wall out of frustration.

This reminds me of my first day using Ubuntu, it took 6 hours to get my wireless working! Now Ubuntu works so well with wireless devices it can link to my home hub during the install. There's a fine line between "Giving plenty of options for a power user" and "Being ****ing unusable for anyone but a power user".

7. przyksquishyValued Senior Member

Messages:
3,203
Have you tried simply using the library just from your current working directory? If it's a Java package then the Java command line tools have a -classpath switch that you can use to tell it explicitly where it should look for packages. It wouldn't be a permanent solution, but at least you could really check that everything is working and that it really is just a matter of putting the library in the right place.

8. AlphaNumericFully ionizedRegistered Senior Member

Messages:
6,702
Right, I've gone through your guide. I had to change .x86_64 to -amd64 in a number of places, like the java directory. My java -versions gives the same as you and the sim-link for default java indeed points to the java-1.6.0-openjdk-amd64, which in turn has a sim-link to java-6-openjdk-amd64 (Jesus Christ, talk about layering on the incomprehensibility!!). I used the rxtx from here as this page suggests, which meant I didn't have to do the bit of code where you started with pwd, but then you point out it didn't work for you anyway, so I did as you did with the manual copy. There's no .la files but there are two .so files. So finally, FINALLY, I can run 'javac TwoWaySerialComm.java' without it spitting an error. It doesn't actually do anything other than return me to the prompt but no error is better than some error.

Thanks so much Chipz. As a thanks if you're ever in the physics/maths forum and want to rant at someone you get one free ad-hom filled swear-fest at a person of your choice. Though I doubt you'd really want to take me up on that offer.....

/edit

For some reason, possibly unrelated because other people have had the problem, after all of this the Arduino IDE stopped recognising the ttyACM0 port the connection is via. It appears in /dev/ but doesn't appear on the port drop down list, only ttyS0 appears. Previously both had and the ttyACM0 is the correct one. After checking my other computers, Windows 7 and Lubuntu 12.04, worked I had to force a sim-link via 'sudo ln -sf /dev/ttyACM0 /dev/ttyS0'. Whether that comes back to bite me in the backside later I have no idea.

Last edited: Oct 1, 2012
9. AlphaNumericFully ionizedRegistered Senior Member

Messages:
6,702
It seems it has.....

When I load Arduino from the terminal it outputs various things to the prompt. Normally it's just the print stuff but now it opens with "RXTX Warning: Removing stale lock file. /var/lock/LCK..ttyS0". I can upload to the Arduino after doing the simlink ACM0 -> S0 but when I try to use the serial monitor to listen to what it sends back the IDE crashes and I get the following error :

RXTX Warning: Removing stale lock file. /var/lock/LCK..ttyS0
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f7dd5209d9d, pid=8206, tid=140178415417088
#
# JRE version: 6.0_24-b24
# Java VM: OpenJDK 64-Bit Server VM (20.0-b12 mixed mode linux-amd64 compressed oops)
# Derivative: IcedTea6 1.11.4
# Distribution: Ubuntu 12.04 LTS, package 6b24-1.11.4-1ubuntu0.12.04.1
# Problematic frame:
#
# /tmp/hs_err_pid8206.log
#
# If you would like to submit a bug report, please include
# instructions how to reproduce the bug and visit:
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
/usr/bin/arduino: line 33: 8206 Aborted (core dumped) java -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel processing.app.Base

Clearly there's some issue between the RXTX library and the ttyACM0 and ttyS0 connections

The simlink fixes one problem and makes another. If I use sudo to open arduino I don't get the lock file problem but it still crashes when I listen to the serial monitor.

10. ChipzBannedBanned

Messages:
838
The problem with file lock is likely due to the lock-file being created with root permissions (sudo), however the IDE is being ran under user mode. Either you will have to make the lock file create in user-mode, change the ownership of the lock, or run the IDE in root. Running it in root shouldn't be a big issue. [edit: ahh, see you did this]

The second problem looks like an ABI issue with the shared objects you used, they were compiled in 2008! That's an eternity in ABI compatibility years. I would suggest just building the shared objects. Additionally these binaries were built for Centos 5.2 [RHEL] rather than any Ubuntu version [Debian]. I'd suggest building them from scratch and then check if you get the same issue. (build instructions were listed above)

11. AlphaNumericFully ionizedRegistered Senior Member

Messages:
6,702
Having reinstalled Ubuntu and tried various permutations of things I can now confirm that if the librxtx*.so and RXTX.jar files are in the ..../jre/lib/... directories then the Arduino IDE doesn't pick up the ttyACM0 port, making the entire endeavour somewhat pointless as I cannot then use the computer the upload programs onto it. Simply doing a 'sudo rm....' on them is enough to fix the problem but restart the original one. Damn.

/edit

It still works if I only put in the *.so files. I'm now thinking that I can just put the .jar file somewhere else, such as the specific project library directory, and be done with it, as that's something I think I saw someone say elsewhere. Is this right? If so where do I put it?

/edit again

Yep, if I copy the RXTX.jar into the ..../ext/ directory then starting up the Arduino IDE fails to find ACM0 and doing it without restart causes it to say "Port not found. Please select a different one" when I click 'upload' and ttyS0 is the only available one. That's the culprit.

/edit AGAIN

Using Netbeans I've loaded up the TwoWaySerialComm.java package and under 'Libraries' it has JDK 1.6 (Default) and under that it has RXTX.jar, along with various others, which seems to suggest it's happily using the JDK version anyway. When I click 'run' it does so without spitting the gnu.io error. Now it spits the following error :

Code:
gnu.io.NoSuchPortException
at gnu.io.CommPortIdentifier.getPortIdentifier(CommPortIdentifier.java:269)
at TwoWaySerialComm.connect(TwoWaySerialComm.java:19)
at TwoWaySerialComm.main(TwoWaySerialComm.java:107)
I'm hazarding a guess that means it isn't sure which port to use but it's at least finding the package.

/edit AGAIN

The .so files bugger up the Serial Monitor so it's not possible for the Arduino IDE to read the information coming out of the board. If I remove them then the serial monitor works properly and I get a nice moving average voltage read out from an IR sensor when I wave my hand in front of it. Except now the java code doesn't compile! Damned if I do, damned if I don't.

12. rpennerFully WiredValued Senior Member

Messages:
4,833
Is Ubuntu running with SELinux turned on? That complicates UNIX owner-group-other permissions into a labyrinth you need an engineer to unwind. For example, if "they" don't think arbitrary programs should access arbitrary serial ports, then "they" don't allow it. (And yes, I am using the "they" of paranoiacs.)

13. AlphaNumericFully ionizedRegistered Senior Member

Messages:
6,702
Right..... right...... right..... It's 12.30am and I've spent 4 hours of my existence today screwing around with this but I think I've got it. I've tried all the various versions of rxtx from the download page, I've tried all combinations of various files added or removed, I've circumvented the version mismatch described here with the 2.2pre version and I think the solution is to not symlink (sorry, kept spelling it with an i) ACM0 and S0 but rather ACM0 and USB0. "But AlphaNumeric....", I hear you ponder, "...why would symlinking one thing the IDE can't see with another which doesn't even exist solve your problem?". Good ****ing question but nevertheless symlinking them makes ttyUSB0 appear on the serial port list, I can upload to it and I can listen to it without the IDE crashing, all while the .so and RXTXcomm.jar files are installed. I came across it because someone else has experienced this fractal cascade of errors.

Woopdee sodding do, I'm going to bed.

14. ChipzBannedBanned

Messages:
838
SELinux is not default for Ubuntu.

15. AlphaNumericFully ionizedRegistered Senior Member

Messages:
6,702
No, I spoke too soon. I get the same Arduino connection errors. Arduino works with the RXTX which is installed when you do sudo apt-get install arduino while any external Java program can't. It seems to be one or the other, not both. Whether this is a 64bit problem, an Ubuntu 12.04 problem or maybe it's because I don't know any Java and hence don't know to do some basic trivial alteration to some menu hidden in the labyrinth that is the Netbeans options but to quote Bill wacko O'Reilly "We'll do it live""..... sorry I mean "****ing thing sucks!". Until I learn enough Java to be able to go deeper into this I'm sticking with the Python example my book has. I know how to do R and Mathematica and Python looks like them enough I can understand it. Java is currently as understandable as Egyptian.....

16. AlphaNumericFully ionizedRegistered Senior Member

Messages:
6,702
As an illustration of how much I hate myself I've continued battling with this, I'm one of those people who can't bear it when something is left unresolved like that. Using a 'slower than a sloth on valium' netbook with 32bit 10.10 installed (so Arduino isn't in the package manager) I went through it again. Installed Arduino and Eclipse, so have java-6-openjdk in jvm/ . If I run Arduino from the terminal then unlike Ubuntu 12.04 some text is printed in the terminal upon loading, specifically the text I get from Netbeans when I try to run the .java file when I have the RXTXcomm.jar file in the jvm/... directory, which shows that its seeing some RXTX file somewhere. Now rather than putting the RXTXcomm.jar file into the jvm/.... directory I add it to the source library list in Netbeans under the test .java file package tree. Running the .java file now spits an error it can't find the libSerial...so file so a step in the right direction, as I can still use the Arduino IDE properly. So I do a search for libSerial....so and low and behold the only copy on the entire computer is in the arduino directory! So Arduino isn't reading it from somewhere on the computer, it has its own copy! A bit of Googling to see if someone else has experienced this missing .so file in Java and, unsurprisingly, people have. Someone says to point Netbean's package VM options to a directory /usr/lib/RXTX/, which should contain the .so files. I don't have such a directory so I sudo mkdir and copy the .so files from the Arduino directory to there. Netbeans, right click, run and FINALLY it runs without an error. Or rather, not an error related to missing libraries. It doesn't find the port because it's looking for the wrong one but it finds all the rxtx crap and the Arduino IDE can still upload and hear the serial port.

I suppose I can take some solace from this, in that I'm now much more familiar with the whole package management thing of Ubuntu and Java than I was a few days ago.....

17. AlphaNumericFully ionizedRegistered Senior Member

Messages:
6,702
And there was one more thing to do. Netbeans wouldn't find the port, no matter what I typed in (ttyUSB0, ttyACM0, COM1 etc) so on a whim I did what I did before and symlinked ttyACM0 to ttyUSB0 and suddenly Netbeans starts outputting "Hello world"! A quick check and the Arduino IDE still works! Everything is working, simultaneously! I don't know how to implement what I did using Eclipse or using the terminal javac but in Netbeans pointing at other locations is straight forward. A learning experience and I managed to do it without yelling too much at the computer at 1am.

/edit

I spoke slightly too soon, as the above is on the 32bit netbook, which is fine. The 64bit version throws a fatal error to do with the librxtxSerial.so file. So I just replaced it with the 32bit version and tried it again. It complains about a version mismatch, which is usually fatal, but it doesn't stop and instead just starts reading the connection properly, printing out the data the Arduino is sending! Have to put in the symlink terminal command into rc.local in order to create it each time the computer starts but that's fine. Aside from the "WARNING : RXTX Version mismatch" initial Netbeans comment it all works on both machines now! Now, where's that soldering iron.....

Last edited: Oct 2, 2012
18. firdroirichA friend of The FriendsRegistered Senior Member

Messages:
565
Hello, I'm glad to see youre having fun with Arduino and Ubuntu