Sunday, April 29, 2007

HTTP POST from a J2ME midlet

NOTE - this blog has now moved to codetrips.com
It is highly recommended that you read the original post there.

-------------

Although there are several examples and tutorials about how to connect to an HTTP server from a J2ME Midlet, I found that they all seem to concentrate a lot on the details of a GET and the POST is always assumed as an "exercise to the reader."

Now, although it is indeed rather straightforward to work out how to send POST requests to a server, I found (the hard way) that there are certain "minor details" that are best not forgotten.

I have thus decided to post here a brief example of how to send a request to a server, using a form-encoded body, so that either a JSP or a servlet can retriever the parameters, but one can benefit from the many advantages of POSTing, as oppposed to GETting.

The source code follows, I believe it is reasonably self-explanatory and comments should guide even the most novice reader through, but please do let me know if you feel that this is still too cryptic:


private void OpenConnection(String server) 
throws java.io.IOException {
// TODO: the actual "resource" part of the URL (/test in this case)
// should be either passed as a parameter or obtained at application
// level (maybe even set up at runtime).
// To test this code, I have written the simple Servlet below, however
// this would also work with a JSP page that echoes back the contents
// in text/plain format for the midlet to display.
//
  String url = "http://"+server+"/test"; 
  byte[] data = null;
  InputStream istrm = null;

  HttpConnection http = (HttpConnection)Connector.open(url);
  http.setRequestMethod(HttpConnection.POST);

  // This allows a JSP page to process the parameters correctly
  // This format (or content type) is the same as when forms are
  // submitted from a web page.
  http.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");

  // You may want to indicate to the server side (eg servlet) that 
  // this request is coming from a specific application.  This would
  // allow generation of appropriate format for the content.
  http.setRequestProperty("User-Agent", "HttpMidlet/0.2");

  // You can send any custom property as part of the HTTP header
  // This would matter for the "infrastructure" code, as opposed
  // to "body parameters" that would matter to the application
  // (eg, user=marco)
  http.setRequestProperty("Custom-Property", 
      "MyCustomProperty/1.0; AnotherProperty/debug_0.1");

  // You MUST create the body of the post BEFORE actually opening
  // the stream, otherwise you won't be able to set the length of
  // the request (see next step).
  // In this example, I am sending coordinates to a mapping server
  String msg = "x="+location.getX()+"&y="+location.getY();

  // THIS is important! without it a JSP won't process the POST data
  // it would also appear that CASE MATTERS (using "Content-Length" -note 
  // the capital 'L'- caused my servlet to return -1 from the 
  //    HttpServletRequest.getContentLenght() method
  http.setRequestProperty("Content-length", ""+msg.getBytes().length);

  // After this point, any call to http.setRequestProperty() will
  // cause an IOException
  OutputStream out = http.openOutputStream();
  out.write(msg.getBytes());
  out.flush();

  if (http.getResponseCode() == HttpConnection.HTTP_OK) {
    int len = (int)http.getLength();
    istrm = http.openInputStream();
    if (istrm == null) {
      log("Cannot open stream - aborting");
      throw new IOException("Cannot open HTTP InputStream, aborting");
    }
    if (len != -1) {
      data = new byte[len];
      int bytesRead = istrm.read(data);
      addProgressMsg("Read "+bytesRead+" bytes");
    } else {
    ByteArrayOutputStream bo = new ByteArrayOutputStream();
      int ch;
      int count = 0;

      // This is obviously not particularly efficient
      // You may want to use a byte[] buffer to read bytes in chunks
      while ((ch = istrm.read()) != -1) {
        bo.write(ch);
        count++;
      }
      data = bo.toByteArray();
      bo.close();
      addProgressMsg("Read "+count+" bytes");
    }
    response = new String(data);
    addProgressMsg("finished");
  } else {
    log("Response: "+http.getResponseCode()+", "+http.getResponseMessage());
    response = null;
    addProgressMsg("failed: "+http.getResponseMessage());
  }
  // This is critical, unless you close the HTTP connection, the application
  // will either be consuming needlessly resources or, even worse, sending
  // 'keep-alive' data, causing your user to foot unwanted bills!
  http.close();
}


To test this code, all that is required is a simple Serlvet that runs, for example, in an Apache Tomcat JSP container:
public class ConnectivityDiag extends HttpServlet {

  private static final String REVISION = "0.0.04";
  private static Logger log;

  public ConnectivityDiag() {
    if (log == null) {
      log = Logger.getLogger(getClass().getName());
      BasicConfigurator.configure();
    }
  }

  public void doGet(HttpServletRequest req, HttpServletResponse res)
      throws ServletException, IOException {
    doProcessRequest(req, res);
  }

  public void doPost(HttpServletRequest req, HttpServletResponse res)
      throws ServletException, IOException {
    doProcessRequest(req, res);
  }

  public void doProcessRequest(HttpServletRequest req, HttpServletResponse res) {
    BufferedReader reader = null;
    Writer writer = null;

    try {
      // Note this call seems to be case-sensitive
      // if the midlet does not set exactly "Content-length"
      // as the property, -1 will be returned
      int len = req.getContentLength();
      String contentType = req.getContentType();
      log.debug("Len: " + len);
      log.debug("Type: "+contentType);

      int bytesRead = 0;
      String remoteHost = req.getRemoteHost();
      String userAgent = req.getHeader("user-agent");
      log.info("Accessed at " + (new Date()) + " from " + remoteHost
          + ", using " + userAgent);

      // This simply echoes the parameters back to the client.
      // A JSP would use these in a  tag
      // and the ${param} macro.
      ArrayList strings = new ArrayList();
      Enumeration e = req.getParameterNames();
      while (e.hasMoreElements()) {
        String param = (String)e.nextElement();
        String value = (String)req.getParameter(param);
        strings.add(param+" = "+value);
      }
      res.setContentType("text/plain");

      // This is a custom property, that the remote client 
      // could query
      res.setHeader("Sent-at", new Date().toString());

      writer = res.getWriter();
      writer.write("Diagnostic servlet - Rev. "+REVISION);
      writer.write("\nBytes received: " + len);
      writer.write("\nFrom: " + remoteHost);
      writer.write("\nUsing: " + userAgent);
      writer.write("\n=== Parameters ===\n");
      for(String s:strings) {
        writer.write(s);
        writer.write('\n');
      }
      writer.write("=== Parameters end here ===\n");
      writer.flush();

    } catch (IOException e) {
      log.error(e.getMessage());
    } finally {
      try {
        if (reader != null)
          reader.close();
        if (writer != null)
          writer.close();
      } catch (IOException ex) {
        log.error(ex.getMessage());
      }
    }
  }
}

22 comments:

  1. Hello,
    I found your blog very useful in creating POST request for my application. Thanks very much for that.
    I have a question though regarding deploying my application on actual phone. When I try to run application (that runs perfectly on emulator)on actual phone, it gives me an error "Error in HTTP operation". Can you give me any suggestion how to solve this problem?

    ReplyDelete
  2. Interesting... what phone model, operating system and J2ME variant was it running?

    I'll tell you beforehand, if it's a Windows Mobile, I'd give up any hope of have anything J2ME-related working, Microsoft's choice of KVM (a very poor implementation by third-party outfit) is really a bare-bones one.

    I have used this very same code (well, a slightly more efficient variation, but essentially all there is to it...) on a few applications I wrote both for ourselves and for our clients, running on anything from Nokia N73/65 and assorted SonyEricsson Wxxx's so I know it does work :-)

    Have you tried to pinpoint at which point in the code the error happens, whether you can open 'ordinary' GET HTTP requests - and, btw, is this happening when you connect over-the-air or when tethered (eg via USB) to a PC?
    If you are connecting via 3G, is the server reachable (eg, can you use the phone's browser to reach it?)

    Welcome to wonderful world of mobile development :-)

    PS - I've now entirely switched to developing for Android (http://code.google.com/android) a much saner approach!

    ReplyDelete
  3. Hi Marco.
    Thanks for replying back. I was able to solve my problem. It turned out that there was nothing wrong with the code, but the access point settings inmy phone were not correct. It s interesting to note though that we need to take care of so many other things while deploying on actual phone than running the code in emulator.
    Thanks again.

    ReplyDelete
  4. can this be used to post a file from my phone? for example, an audio file in .amr format?

    how do i set the path to it.

    Thanks

    ReplyDelete
  5. Well, in principle it could - but it's not a matter of 'setting the path' :-)

    You would have to acquire the file's contents (say, for an .AMR audio segment, the bytes that compose the audio) by whatever means the underlying platform provides (not all J2ME implementation support the File I/O API - the 'standard' data storage access in J2ME is the Record Management System or RMS).

    Those bytes then could be written using the methods described in the article, and you would have to set the ContentPath header to correctly indicate the MIME type.

    In fact, the code shown, was actually used to upload both audio and location data to a server.

    ReplyDelete
  6. Hi,
    I'm in trouble with accessing this page using JSON posting from my j2me midlet:
    http://www.vcast.it/faucetpvr/

    Here you find specifications:
    http://www.vcast.it/TestS/?xml=api_specification.xml

    (point 3, inserting new recordings)


    I keep getting "Bad request" response, can you help?

    ReplyDelete
  7. Sorry, never mind previous comment: I didn't notice the warning about case sensitiveness of "Content-length" field!

    Thank you very much.

    ReplyDelete
  8. Really nice work done...

    ReplyDelete
  9. Thanks a lot! This post was very helpful for me

    ReplyDelete
  10. This comment has been removed by the author.

    ReplyDelete
  11. J2ME beginner , Thanks a lot. It helped me too.

    ReplyDelete
  12. Thanks a lot , it helped me too. Will check on real mobile device.

    ReplyDelete
  13. I reproduce his example but I had no success with the application embedded onto the device. Message (IOException: Error in HTTP operarion)
    In the emulator works well. What can it be?

    http://www.forum.nokia.com/devices/1680_classic/

    ReplyDelete
  14. Great tutorial... Now I am in the need of HttpsConnection for secure http... let us know if anyone already did some work...

    ReplyDelete
  15. I have a midlet that runs fine on Nokia, SonyEricsson and LG, but in Samsung (ex. GT-E2121 model), I run it one time and works ok but If I close it and run again it don't send data to the web server and get no response from getresponsecode(), not even null, just empty. I have to turn the phone off and on and its works again. I use httpconnection with POST method. The G appears when it first connect. But when I start the application for a second time, it not connect again. Any idea? Thanks a lot!

    ReplyDelete
  16. Hi SG,

    it's now several years since I last did some serious J2ME development (gotta get with the Android program!) but from you say, I'd guess it has something to do with the HTTP connection not being closed, perhaps the stream is not closed, or the server never closes the stream to the device.

    Do you process the response from the server? Open an InputStream (or an OutputStream?) do they get closed?

    Very difficult to say without looking at the code and testing out with the device, but those are the crannies I'd go looking for: clearly the Samsung driver is not timing out, or maybe it just keeps the connection open to minimize use of sockets, but then if one does not close it, it just hangs.

    ReplyDelete
  17. error in http operation.. :( could u plz help me out.. it works fine in emulator but on real device it shows the above error..
    thank u very much sir.. :)

    ReplyDelete
  18. From your report is not even clear where do you get the error message (is it a log, an HTTP return code --if so what actual code you got, an error dialog...) and what were you trying to do.

    For example, did you try to open the web browser on the phone and point it to the same server? Did it work?
    Did you try the same code on another phone / network / SIM? to another URL?

    If it works fine on the device, my guess is that you're either hitting a firewall issue, or the phone manufacture and/or carrier may have disabled access to certain IPs, or whatever.
    (this sounds silly, but are you trying to hit http://localhost? that would explain it working in emulator but not on the phone)

    May I suggest you have a look here:
    http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

    ReplyDelete
  19. "http.setRequestProperty("User-Agent", "HttpMidlet/0.2");"

    What if you don't know the User-Agent?
    I mean, i want to send the actual device user-agent from application?

    ReplyDelete
  20. the easiest way is to connect to a server you manage (eg, an instance of Tomcat/Apache running on your box) and see what User-Agent you receive when browsing to that server from the phone's browser

    there is no such a thing as a "device user-agent" - it is application specific (so FFox will send a different UA than, say, Opera, than the native device's browser)

    another option is to just run a simple server on your device (using, for example, Socket.accept()) on a given port (say, 8081) and then browsing on your device's browser to http://localhost:8081 - you can then 'sniff' whatever the browser is sending, including the HTTP headers.

    The real question, though, is why do you want to do that?
    (oh, yes - the less-than-technical alternative, obviously, is to Google for it :)

    ReplyDelete
  21. people like you are angels!
    i get rid of some sort of bad problems after reading your tutorial.
    now it works fine!
    i hope you best days :)

    ReplyDelete
  22. Thanks, Sina -- whilst I can't quite see myself like an 'angel', it's good to know this has been helpful to you :)

    ReplyDelete