Wednesday, September 8, 2010
Saturday, July 3, 2010
Networking
When people hear the word "Networking", they mostly think of social networking like Facebook, Linkedin,and Twitter..I would think the same way honestly since they are really in our lives now..But here what I meant with networking was actually TCP/IP networking. I know it is not that fun..
What is TCP and what is IP? And what about UDP..I am sure interested readers have once in a life time tried ping and traceroute. Is it UDP again? Or something different such as ICMP maybe.
What are all these protocols?
How do you think the requests we send from our browsers get to the destination server, retrieve the data and bring it to us, clients? Have you ever thought what the infrastructure is like? As I am very curious about the organization of a computing system and the hardware abstraction via software, I am very interested in seeing what is going on in the Network layers so that the data can go back and forth. We do video chat, and web browsing, we stream videos, audio files..So how does that work? The OS handles all those things for us. And the socket API is the greatest factor in its success. But how does it work? If you want to observe how it works in real-time, I would recommend you use Wireshark tool. This is a free software and also little bit dangerous :) But if you know what you need, you can really get with this tool.
I will try to explain what you are going to see in Wireshark when you follow the protocols and the information in each packet using those protocols. The protocols you will see mostly are UDP, TCP, ICMP, DNS, DHCP, SSDP..I am sure most of you know DNS. DNS is like a name lookup in an address book. How it works? The DNS translates the name to an 32 bit IP address. How your client knows the DNS server is another issue. This information is mostly hardcoded in your browser.
DHCP, another important concept of networking. It is Dynamic Host Configuration Protocol. When a computer boots-up, it broadcasts its address in the network. And the DHCP in the network assigns an IP address to this physical address. This is what DHCP does.
UDP and TCP are transport layer protocols, which are connectionless and connection-oriented respectively. If you are familiar with the 3-way handshake connection establishment and 4-way teardown in TCP, the Wireshark helps you understand how, for example http works. TCP is basically a reliable, connection oriented protocol that ensures the data delivery. But it is costly to the system. To understand what I mean, you need to have a good understanding of the TCP mechanism, acknowledgements, flow control, and re-transmission issues. There is a trade-off between UDP and TCP. If you care about the speed rather than accurate delivery than you would probably prefer UDP. What happens in TCP is that when a TCP client starts to establish a connection, an active open, it sends a SYN package to the destination server, thus starts the 3-way handshake. The server transitions to the passive open state from the listening state and sends a SYN,ACK package to the client acknowledging that it received the SYN package. (Client and server has initial sequence numbers that they use in the first SYN and SYNACK packages) The client sends the ACK package and then sends the first data segment to the client. This state is Established state where data exchange occurs between the client and the server. One side has to send FIN to end the connection. Generally the client sends the first FIN package and starts the active close while the receiving side, server, switches to the passive close state. If you search for TCP state machine, especially read the RFC793, I think you will be more clear. ANd the passive-closing side sends a FIN after sending the ACK for the last received FIN. After receiving the ACK for the last FIN sent, it transitions to the Listen state. The states are very slightly different for the server and client, but I can say almost the same. So this is what makes TCP expensive. It makes sense that the video streaming applications use UDP rather than TCP. I can not think of 3-way handshake at the beginning of connection and re-transmission for all lost packages sent from Youtube servers. They would crash probably. Even though UDP is fast, the receiver still uses buffering mechanism in order to display the video in the correct order and with least delay. The speed is very important and, there is also RTP/RTCP protocols that can patch for the lost data package for audio applications. There is one more thing here.Have you ever thought how it happens? How does the audio and video syncronize..Wouldn't it be weird if what we hear and see were not synchronized? RTCP, here it comes :) The user does not really understand as long as there are not very long delays actually..
Fianlly, there are some applications which use ICMP, IGMP and passes over the transport layer. The application directly deals with the IP layer. And you have to be superuser in order to directly access to the IP layer. This is tricky actually because you can easily crash a server if you know how to play with ICMP. Because it provides you the ability to change the IP address of the sender, and you can send many request to too many computers, and route these replies to another computer that you use the IP address of. But rather than for bad intentions, ICMP is used to troubleshoot the network with traceroute and ping. We are engineers, we need to serve for good. Engineering ethics :)
Well that is all for now, I was planning to write more technical and implementation details rather than just describing the concept but I will do it in my next post. How is this implemented? Good question. Please read the RFCs and explore the Linux Socket API functions and structures.
They are designed to implement all this theory.
Good night.Enjoy the sunny sunday..
What is TCP and what is IP? And what about UDP..I am sure interested readers have once in a life time tried ping and traceroute. Is it UDP again? Or something different such as ICMP maybe.
What are all these protocols?
How do you think the requests we send from our browsers get to the destination server, retrieve the data and bring it to us, clients? Have you ever thought what the infrastructure is like? As I am very curious about the organization of a computing system and the hardware abstraction via software, I am very interested in seeing what is going on in the Network layers so that the data can go back and forth. We do video chat, and web browsing, we stream videos, audio files..So how does that work? The OS handles all those things for us. And the socket API is the greatest factor in its success. But how does it work? If you want to observe how it works in real-time, I would recommend you use Wireshark tool. This is a free software and also little bit dangerous :) But if you know what you need, you can really get with this tool.
I will try to explain what you are going to see in Wireshark when you follow the protocols and the information in each packet using those protocols. The protocols you will see mostly are UDP, TCP, ICMP, DNS, DHCP, SSDP..I am sure most of you know DNS. DNS is like a name lookup in an address book. How it works? The DNS translates the name to an 32 bit IP address. How your client knows the DNS server is another issue. This information is mostly hardcoded in your browser.
DHCP, another important concept of networking. It is Dynamic Host Configuration Protocol. When a computer boots-up, it broadcasts its address in the network. And the DHCP in the network assigns an IP address to this physical address. This is what DHCP does.
UDP and TCP are transport layer protocols, which are connectionless and connection-oriented respectively. If you are familiar with the 3-way handshake connection establishment and 4-way teardown in TCP, the Wireshark helps you understand how, for example http works. TCP is basically a reliable, connection oriented protocol that ensures the data delivery. But it is costly to the system. To understand what I mean, you need to have a good understanding of the TCP mechanism, acknowledgements, flow control, and re-transmission issues. There is a trade-off between UDP and TCP. If you care about the speed rather than accurate delivery than you would probably prefer UDP. What happens in TCP is that when a TCP client starts to establish a connection, an active open, it sends a SYN package to the destination server, thus starts the 3-way handshake. The server transitions to the passive open state from the listening state and sends a SYN,ACK package to the client acknowledging that it received the SYN package. (Client and server has initial sequence numbers that they use in the first SYN and SYNACK packages) The client sends the ACK package and then sends the first data segment to the client. This state is Established state where data exchange occurs between the client and the server. One side has to send FIN to end the connection. Generally the client sends the first FIN package and starts the active close while the receiving side, server, switches to the passive close state. If you search for TCP state machine, especially read the RFC793, I think you will be more clear. ANd the passive-closing side sends a FIN after sending the ACK for the last received FIN. After receiving the ACK for the last FIN sent, it transitions to the Listen state. The states are very slightly different for the server and client, but I can say almost the same. So this is what makes TCP expensive. It makes sense that the video streaming applications use UDP rather than TCP. I can not think of 3-way handshake at the beginning of connection and re-transmission for all lost packages sent from Youtube servers. They would crash probably. Even though UDP is fast, the receiver still uses buffering mechanism in order to display the video in the correct order and with least delay. The speed is very important and, there is also RTP/RTCP protocols that can patch for the lost data package for audio applications. There is one more thing here.Have you ever thought how it happens? How does the audio and video syncronize..Wouldn't it be weird if what we hear and see were not synchronized? RTCP, here it comes :) The user does not really understand as long as there are not very long delays actually..
Fianlly, there are some applications which use ICMP, IGMP and passes over the transport layer. The application directly deals with the IP layer. And you have to be superuser in order to directly access to the IP layer. This is tricky actually because you can easily crash a server if you know how to play with ICMP. Because it provides you the ability to change the IP address of the sender, and you can send many request to too many computers, and route these replies to another computer that you use the IP address of. But rather than for bad intentions, ICMP is used to troubleshoot the network with traceroute and ping. We are engineers, we need to serve for good. Engineering ethics :)
Well that is all for now, I was planning to write more technical and implementation details rather than just describing the concept but I will do it in my next post. How is this implemented? Good question. Please read the RFCs and explore the Linux Socket API functions and structures.
They are designed to implement all this theory.
Good night.Enjoy the sunny sunday..
Sunday, June 13, 2010
Memory Types
One of the most important concept in every computing system, the Memory.
Today in Personal Computers there is plenty of memory space whereas the embedded systems still lack enough memory space. Power consumption, memory and disk space, size are the limitations of the embedded systems. For that reason, programming accordingly is crucial for the system.
We are not in the era when 64KB memory was a great success. The size of memory goes up to GigaBytes but still there are important issues in memory design and use.types of memory? Which one is the best fit for your system?? To be able to answer this question, you need to know the types, the volatility, addressing, write-ability issues in a memory chip. Today I will discuss about the memory types existing in embedded systems.
RAM and ROM are the basic types however there are hybrid types as well. RAM memories are volatile chips, which means if the power is off, you lose all the data in that location. If you are using a SRAM, which is a static RAM and mostly used as CPU cache, you can keep the data there as log you supply with the power. But if you are using a DRAM, the data is refreshed periodically and you cant keep the data forever. A DRAM is nothing if you do not have a DRAM controller for it. That is why a memory controller is important in the proper functioning of a DRAM. Unlike RAMs, ROMs are stable memory devices. Even if the power is off, the data still remains in the memory chip. But once the data is written there, you cant easily erase or re-write the data. To overcome the difficulties of ROM and RAM type memories, the hardware engineers developed hybrid types of memory, which are re-writable but non-volatile. The most common ones are NVRAM and Flash memories. I have never used a NVRAM but I am currently programming a data flash to store some temperature data I read in my room. As I make progress in this project, I will keep you updated with this blog as well :-) This is just a very basic information about memories currently used.
Have a GREAT Sunday!
And yes, I didnt forget to write about Makefiles...
Today in Personal Computers there is plenty of memory space whereas the embedded systems still lack enough memory space. Power consumption, memory and disk space, size are the limitations of the embedded systems. For that reason, programming accordingly is crucial for the system.
We are not in the era when 64KB memory was a great success. The size of memory goes up to GigaBytes but still there are important issues in memory design and use.types of memory? Which one is the best fit for your system?? To be able to answer this question, you need to know the types, the volatility, addressing, write-ability issues in a memory chip. Today I will discuss about the memory types existing in embedded systems.
RAM and ROM are the basic types however there are hybrid types as well. RAM memories are volatile chips, which means if the power is off, you lose all the data in that location. If you are using a SRAM, which is a static RAM and mostly used as CPU cache, you can keep the data there as log you supply with the power. But if you are using a DRAM, the data is refreshed periodically and you cant keep the data forever. A DRAM is nothing if you do not have a DRAM controller for it. That is why a memory controller is important in the proper functioning of a DRAM. Unlike RAMs, ROMs are stable memory devices. Even if the power is off, the data still remains in the memory chip. But once the data is written there, you cant easily erase or re-write the data. To overcome the difficulties of ROM and RAM type memories, the hardware engineers developed hybrid types of memory, which are re-writable but non-volatile. The most common ones are NVRAM and Flash memories. I have never used a NVRAM but I am currently programming a data flash to store some temperature data I read in my room. As I make progress in this project, I will keep you updated with this blog as well :-) This is just a very basic information about memories currently used.
Have a GREAT Sunday!
And yes, I didnt forget to write about Makefiles...
hello world, merhaba dünya!!
For those who want to have some fun(!) in these boring days, I would recommend you start reviewing your first days of embedded programming..Saying hello to our cruel world as every new-born programmer does :) I wont talk about how to write a Hello World program in C but rather about how this runs on your system..Yes the build process..When you write a program, as you understand it, the machine has to be able to understand and interpret the code. So how does it work? What does the OS do so that you see the execution results on the screen? It uses some compiler tools ---->Compile - Link- Relocate and Load. I am sure many of you are familiar with this process. Especially if you are using GNU tools like gcc, as, ld!!! If you did not use it before I recommend that you start soon since it is open source, easy to use.
Well, when you develop software, the very first thing you need to do is compile each source file..If the compilation process is successful, then you will have object files (.o extensions). All the source files must be assembled to an object file. The second step is to produce one object file. This file will be the combination of all the object files in one single file. Every object file has text, data, and bss segments. The machine code resides in the text segment, the initialized variables go to the data segment and unitialized variables go to the bss segment. In the final object file these segments include these segments of all the object files. After this step, the relative addresses must be assigned in the code, which is relocation..Relocation is important since when the code is loaded into the physical memory, the relative offsets play an important role. Because the code will not be loaded to the 0th memory location but to any empty location with the relative offsets. This final file with the relocation information is ready to run in the memory. As I mentioned above, these steps can be fulfilled with GNU tools. GNU gcc is a compilation tool. You can compile your code with a command such as: "gcc -g -c -Wall -I../include source_files.c". -g flag is for debug information, -c tells the compiler not to link files, and -Wall flag means turn the warnings on, and -I../include tells the compiler to look in the include directory for the header files..I also want to say that, for an embedded systems, you will be using a cross-compiler and there must be a toolchain that converts the code into your embedded target board machine code. I am using ARM board and in my case, I use arm-elf-gcc instead of native gcc. Why am I doing this? Because it doesnt make sense to compile the code on the embedded target since it is already resource restricted. For that reason we use host platforms and develop the code using a cross compiler. After compilation, we need to link and locate. GNU has the ld tool. A sample command can be such as: "arm-elf-ld -Map source.map -T arm.ld -o source.exe source_files.o". -Map flag is used to generate a map file. arm.ld is a linker script which gives information about the ram, rom start adress, memory sizes as well as text, data, and bss start address. -o flag tells to link the object files and make an executable file, which is sources.exe in this case. Now the code is ready to run on your embedded system as it is transfered to the board.
Rather than doing these steps manually, we can also use the make utility. It simplifes the compilation process and used in every project. But I will talk about it in my next post..
For now have a good night.
Well, when you develop software, the very first thing you need to do is compile each source file..If the compilation process is successful, then you will have object files (.o extensions). All the source files must be assembled to an object file. The second step is to produce one object file. This file will be the combination of all the object files in one single file. Every object file has text, data, and bss segments. The machine code resides in the text segment, the initialized variables go to the data segment and unitialized variables go to the bss segment. In the final object file these segments include these segments of all the object files. After this step, the relative addresses must be assigned in the code, which is relocation..Relocation is important since when the code is loaded into the physical memory, the relative offsets play an important role. Because the code will not be loaded to the 0th memory location but to any empty location with the relative offsets. This final file with the relocation information is ready to run in the memory. As I mentioned above, these steps can be fulfilled with GNU tools. GNU gcc is a compilation tool. You can compile your code with a command such as: "gcc -g -c -Wall -I../include source_files.c". -g flag is for debug information, -c tells the compiler not to link files, and -Wall flag means turn the warnings on, and -I../include tells the compiler to look in the include directory for the header files..I also want to say that, for an embedded systems, you will be using a cross-compiler and there must be a toolchain that converts the code into your embedded target board machine code. I am using ARM board and in my case, I use arm-elf-gcc instead of native gcc. Why am I doing this? Because it doesnt make sense to compile the code on the embedded target since it is already resource restricted. For that reason we use host platforms and develop the code using a cross compiler. After compilation, we need to link and locate. GNU has the ld tool. A sample command can be such as: "arm-elf-ld -Map source.map -T arm.ld -o source.exe source_files.o". -Map flag is used to generate a map file. arm.ld is a linker script which gives information about the ram, rom start adress, memory sizes as well as text, data, and bss start address. -o flag tells to link the object files and make an executable file, which is sources.exe in this case. Now the code is ready to run on your embedded system as it is transfered to the board.
Rather than doing these steps manually, we can also use the make utility. It simplifes the compilation process and used in every project. But I will talk about it in my next post..
For now have a good night.
Sunday, May 23, 2010
When starting a blog..
Welcome to my blog..This blog is dedicated to share and discuss information about embedded systems develoment, Linux, and basic C..
Hope you find it informative.
Have fun!
Hope you find it informative.
Have fun!
Subscribe to:
Posts (Atom)