Projects & news from nuelectronics.com

Project note — Ethernet Shield for Arduino - a Web client example

This project shows how to implement a distributed sensor network by using the arduino and the ethernet shield.

1. Web Client concept

Since I published ethernet Shield web-server project note a few months ago [link], I’ve got a few emails about how to use the Ethernet shield as a web client. In a sense, a web client application is more appropriate for a small device such as the Ethernet shield on Arduino.

Based on the TCP/IP protocol, it is the web client (such as a web browser) who initializes the TCP connections, requests data from or sends data to the web server. Therefore the ethernet shield running in web client mode can be used as a distributed sensor node in the network. It can send periodic or interrupt driven sensor data to a web server. The web server can then record, process and show the sensor data from one or multiple web client.

In this project, a web client application is devloped for the ethernet shield, which can send periodic data (such as temperature reading) or spontaneous data (such as infrared sensor data or switch press) to a webserver. PHP scripts that used to save and show the data received on the websever are also developed.

2. Web Client TCP/IP implementation

The Web client TCP/IP protocol is implemented in the client_process() routine, which is called in the main Arduino sketch loop() routine. The client process uses a state machine to control the TCP/IP protocol as shown in the following code segment –

void client_process ( void )
{
    uint16_t plen;
	uint8_t i;
    if (client_data_ready == 0)  return;     // nothing to send

	if(client_state == IDLE){   // initialize ARP
       es.ES_make_arp_request(buf, dest_ip);

	   client_state = ARP_SENT;
	   return;
	}

	if(client_state == ARP_SENT){

        plen = es.ES_enc28j60PacketReceive(BUFFER_SIZE, buf);

		// destination ip address was found on network
        if ( plen!=0 )
        {
            if ( es.ES_arp_packet_is_myreply_arp ( buf ) ){
                client_state = ARP_REPLY;
				syn_ack_timeout=0;
				return;
            }

		}
	        delay(10);
		syn_ack_timeout++;

		if(syn_ack_timeout== 100) {  //timeout, server ip not found
			client_state = IDLE;
			client_data_ready =0;
			syn_ack_timeout=0;
			return;
		}
    }

 // send SYN packet to initial connection
	if(client_state == ARP_REPLY){
		// save dest mac
		for(i=0; i<6; i++){
			dest_mac[i] = buf[ETH_SRC_MAC+i];
		}

        es.ES_tcp_client_send_packet (
                       buf,
                       80,
                       1200,
                       TCP_FLAG_SYN_V,                 // flag
                       1,                                              // (bool)maximum segment size
                       1,                                              // (bool)clear sequence ack number
                       0,                                              // 0=use old seq, seqack : 1=new seq,seqack no data : new seq,seqack with data
                       0,                                              // tcp data length
		      dest_mac,
		      dest_ip
                       );

		client_state = SYNC_SENT;
	}
  // get new packet
  if(client_state == SYNC_SENT){
    plen = es.ES_enc28j60PacketReceive(BUFFER_SIZE, buf);

       // no new packet incoming
    if ( plen == 0 )
    {
        return;
    }

       // check ip packet send to avr or not?
       // accept ip packet only
    if ( es.ES_eth_type_is_ip_and_my_ip(buf,plen)==0){
		return;
    }

       // check SYNACK flag, after AVR send SYN server response by send SYNACK to AVR
    if ( buf [ TCP_FLAGS_P ] == ( TCP_FLAG_SYN_V | TCP_FLAG_ACK_V ) )
    {

               // send ACK to answer SYNACK

               es.ES_tcp_client_send_packet (
                       buf,
                       80,
                       1200,
                       TCP_FLAG_ACK_V,                 // flag
                       0,                                              // (bool)maximum segment size
                       0,                                              // (bool)clear sequence ack number
                       1,                                              // 0=use old seq, seqack : 1=new seq,seqack no data : new seq,seqack with data
                       0,                                              // tcp data length
						dest_mac,
						dest_ip
                       );
               // setup http request to server
               plen = gen_client_request( buf );
               // send http request packet
               // send packet with PSHACK
               es.ES_tcp_client_send_packet (
                                       buf,
                                       80,                                             // destination port
                                       1200,                                   // source port
                                       TCP_FLAG_ACK_V | TCP_FLAG_PUSH_V,                        // flag
                                       0,                                              // (bool)maximum segment size
                                       0,                                              // (bool)clear sequence ack number
                                       0,                                              // 0=use old seq, seqack : 1=new seq,seqack no data : >1 new seq,seqack with data
                                       plen,                           // tcp data length
                                       dest_mac,
									   dest_ip
									   );
               return;
       }
       // after AVR send http request to server, server response by send data with PSHACK to AVR
       // AVR answer by send ACK and FINACK to server
       if ( buf [ TCP_FLAGS_P ] == (TCP_FLAG_ACK_V|TCP_FLAG_PUSH_V) )
       {
               plen = es.ES_tcp_get_dlength( (uint8_t*)&buf );

               // send ACK to answer PSHACK from server
               es.ES_tcp_client_send_packet (
                                       buf,
                                       80,                                             // destination port
                                       1200,                                   // source port
                                       TCP_FLAG_ACK_V,                  // flag
                                       0,                                              // (bool)maximum segment size
                                       0,                                              // (bool)clear sequence ack number
                                       plen,                                           // 0=use old seq, seqack : 1=new seq,seqack no data : >1 new seq,seqack with data
                                       0,                              // tcp data length
				      dest_mac,
				      dest_ip
               );;
               // send finack to disconnect from web server

               es.ES_tcp_client_send_packet (
                                       buf,
                                       80,                                             // destination port
                                       1200,                                   // source port
                                       TCP_FLAG_FIN_V|TCP_FLAG_ACK_V,                  // flag
                                       0,                                              // (bool)maximum segment size
                                       0,                                              // (bool)clear sequence ack number
                                       0,                                           // 0=use old seq, seqack : 1=new seq,seqack no data : >1 new seq,seqack with data
                                       0,
										dest_mac,
										dest_ip
				);

               return;

       }
       // answer FINACK from web server by send ACK to web server
       if ( buf [ TCP_FLAGS_P ] == (TCP_FLAG_ACK_V|TCP_FLAG_FIN_V) )
       {
               // send ACK with seqack = 1
               es.ES_tcp_client_send_packet(

                                       buf,
                                       80,                                             // destination port
                                       1200,                                   // source port
                                       TCP_FLAG_ACK_V,                 // flag
                                       0,                                              // (bool)maximum segment size
                                       0,                                              // (bool)clear sequence ack number
                                       1,                                              // 0=use old seq, seqack : 1=new seq,seqack no data : >1 new seq,seqack with data
                                       0,
									   dest_mac,
									   dest_ip
				);
			client_state = IDLE;		// return to IDLE state
			client_data_ready =0;		// client data sent
		}
  }
}

The client data is generated in the gen_client_request routine. The following example shows that a temperature reading is filled into the http buffer. The keyword “pwd”, “client” and “status” are used for the server php script to identify the data that client sends.

uint16_t gen_client_request(uint8_t *buf )
{
	uint16_t plen;
	byte i;

	plen= es.ES_fill_tcp_data_p(buf,0, PSTR ( "GET /ethershield_log/save.php?pwd=secret&client=" ) );
        for(i=0; client_ip[i]!='\0'; i++){
            buf[TCP_DATA_P+plen]=client_ip[i];
            plen++;
        }
        plen= es.ES_fill_tcp_data_p(buf,plen, PSTR ( "&status=temperature-" ) );
        for(i=0; sensorData[i]!='\0'; i++){

                buf[TCP_DATA_P+plen]=sensorData[i];
                plen++;
        }	

	plen= es.ES_fill_tcp_data_p(buf, plen, PSTR ( " HTTP/1.0\r\n" ));
	plen= es.ES_fill_tcp_data_p(buf, plen, PSTR ( "Host: 192.168.1.4\r\n" ));
	plen= es.ES_fill_tcp_data_p(buf, plen, PSTR ( "User-Agent: AVR ethernet\r\n" ));
        plen= es.ES_fill_tcp_data_p(buf, plen, PSTR ( "Accept: text/html\r\n" ));
	plen= es.ES_fill_tcp_data_p(buf, plen, PSTR ( "Keep-Alive: 300\r\n" ));
	plen= es.ES_fill_tcp_data_p(buf, plen, PSTR ( "Connection: keep-alive\r\n\r\n" ));

	return plen;
}

3. Web Server and PHP scripts

To test the web client application, I set up a web server in my home LAN environment. I used the wampserver package, which has Apache, PHP and MYSQL, in the windows. The installation is straightforward, after installation you need to allow the client ip address to access your apache webserver by modifying the httpd.conf file. For me, I just enable all my local ip addresses as –

    #
    # Possible values for the Options directive are "None", "All",
    # or any combination of:
    #   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
    #
    # Note that "MultiViews" must be named *explicitly* --- "Options All"
    # doesn't give it to you.
    #
    # The Options directive is both complicated and important.  Please see
    # http://httpd.apache.org/docs/2.2/mod/core.html#options
    # for more information.
    #
    Options Indexes FollowSymLinks

    #
    # AllowOverride controls what directives may be placed in .htaccess files.
    # It can be "All", "None", or any combination of the keywords:
    #   Options FileInfo AuthConfig Limit
    #
    AllowOverride all

    #
    # Controls who can get stuff from this server.
    #
#   onlineoffline tag - don't remove
    Order Deny,Allow
#    Deny from all
    Allow from 127.0.0.1 192.168.1

Two PHP script are developed for saving and showing the web client data. The save.php is used to save web client data to a data.txt file. It looks for the keywords “pwd”, “client” and “status” in the http data sent by the client, and then record them with timestamps.

<?php
function get_val ( $val )
{

		return $_GET[$val];

}

$password = get_val ( 'pwd' );

if ( $password != 'secret' )
{
	echo "<font color=red><b>Access denied!!!</b></font>";
	exit;
}

$client = get_val('client');
if ( $client == '' )
{
	echo "<font color=red><b>Wrong client number!!!</b></font>";
	exit;
}

$status = get_val ( 'status' );

if ( $status == '' )
{
	echo "<font color=red><b>Wrong client status!!!</b></font>";
	exit;
}

$filename = "./data.txt";
$time = date ("Y-n-j H:i:s");

$string = $client . '|' .$time.';'.$status."\r\n";
$a = fopen("$filename", "a+");
fputs($a, $string);
fclose($a);
?>

The index.php script dynamically reads data from the data.txt file and shows them in a table format. It can be accessed for a web browser as shown below.

4. Software structure

A new version of ethernet shield software is developed based on tuxgraph’s code and avrnet code. The software is in the form of Arduino library and completes with a few examples, which consists of -

• etherShield.cpp - a wrapper cpp file, as an Arduino library interface.
• ip_arp_udp_tcp.c - the IP, ARP, UDP and TCP protocol implementation, now with new web client founctions.
• enc28j60.c - ENC28J60 SPI routines
• net.h - network protocol definitions.
• /examples -
  • etherShield_ping - the PING example
  • etherShield_web_temperature - a Web server temperature sensor example
  • etherShield_web_switch - a Web server led switch exampe
  • etherShield_web_client - a Web client example that sends periodic temeperature sensor reading.
  • etherShield_web_client1 - a Web client example that sends spontaneous infrared sensor stauts

5. Links & Downloads

• Arduino Ethernet Shield schematic V1.0
• Arduino ethershield library - new version with web client funtions & examples
• Microchip ENC28J60 datasheet

Tags: Arduino, Arduino projects, ethernet, php, sensor network, web client

This entry was posted on Monday, August 25th, 2008 at 5:47 pm and is filed under Arduino projects. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.