Tuesday, February 4, 2014

Linux Addressing

  • There are two kinds of address used in Linux. 1. Virtual address( logical address)  and 2. Physical address.
  • Physical addresses are the addresses of the contents of the RAM.
  • Virtual addresses as the name says are virtual, i.e. they do not point to any address in RAM directly but need to be converted into physical address by MMU at run time.
  • For a 32 bit system there can be 2^32=4GB virtual addresses possible.
  • Each process is given a separate virtual address space so that they feel as if they have the access to the full RAM and each process remain independent of each other.
  • Virtual address is divided into two parts  1. User space virtual addresses  2. Kernel space virtual addresses.

  • The user space can be given upper 3 GB (0xc0000000 .. 0xffffffff ) and kernel space the lower 1 GB ( 0x0... 0xbfffffff ) also but the diagram shows the general way of doing it.
  • Virtual memory is not really used (committed) until actually used. This means when you allcoate memory, you get an "IOU" or "promise" for memory, but the memory only gets consumed when you actually use the memory, as in write some value to it.
  • A process running in user space may need to request some functions from kernel(ex. syscalls), so kernel image is fully mapped to the Kernel Virtual Space.
  • Suppose a user space application makes use of libc, then libc should also be mapped to the virtual address space of the process 
  • This mapping is kept permanent till the process is terminated.
  • Since the kernel needs to access each and every physical memory so the entire physical memory must be mapped to the kernel virtual space (1 GB).
  • The mapping is one two one.
  • For the mapping to be truly one to one the RAM has to be of 1GB, but this is not the case always.
  • We will consider two cases 1. RAM is less than 1 GB, say 512MB and 2. when the RAM is more than 1 GB say 3GB.
     process address space(virtual address space)

    4GB +---------------+
        |     512MB     |
        +---------------+ <------+     physical memory               
        |     512MB     |        | 
    3GB +---------------+ <--+   +---> +------------+ 
        |               |    |         |   512 MB   |
        |     /////     |    +-------> +------------+
        |               |     
    0GB +---------------+     

  • When the RAM is 512MB then whole of RAM is linearly mapped to the 512MB of the lower 512 MB of the kernel virtual space.
  • The virtual address space of Kernel is divided into two parts 1. Low memory 2. High Memory.
  • The first 896MB constitute the low memory region or LOWMEM.
  • The top 128 MB is called high memory region or HIGHMEM.
  • The first 16MB of LOWMEM is reserved for DMA usage.
  • There are some zones of physical memory:
  • ZONE_DMA - Contains page frames of memory below 16 MB
  • ZONE_NORMAL - Contains page frames of memory at and above 16 MB and below 896 MB
  • ZONE_HIGHMEM - Contains page frames of memory at and above 896 MB
  • So, if you have 512 MB, your ZONE_HIGHMEM will be empty, and ZONE_NORMAL will have 496 MB of physical memory mapped.

                                       physical memory
       process address space    +------> +------------+
                                |        |  3200 M    |
                                |        |            |
    4GB +---------------+ <-----+        |  HIGH MEM  |
        |     128 MB    |                |            |
        +---------------+ <---------+    |            |
        +---------------+ <------+  |    |            | 
        |     896 MB    |        |  +--> +------------+         
    3GB +---------------+ <--+   +-----> +------------+ 
        |               |    |           |   896 MB   |
        |     /////     |    +---------> +------------+
        |               |     
    0GB +---------------+     

  • As we can see from the diagram the physical memory above 896MB is mapped to the 128MB of the HIGHMEM region.
  • When we request a large chunk of memory from physical space using vmalloc() then we get it from HIGHMEM also ,that's why the physical pages that we get are not contiguous.
  • For a user-space application, if you print out the pointer address you defined, it should be one of the virtual addresses out of the (0-3GB) range.

  • What about kernel? what if print a pointer address in kernel? is it always from kernel address space? The answer is no ... since kernel can access user space address, depending on the pointer, it can be from either.
  • To distinguish between kernel space and user space address is easy, if it fall into 0-3GB, then it is from user-space, otherwise, it is from kernel.The programmer sees virtual address only. 

  • To make things clear from user point of view lets see the user space addresses after compiling the program.
+----------------------+ <--- 0xBFFF FFFF (=3GB)
    | environment variable |
    |----------------------| <--- 0xBFFF FD0C 
    | stacks (down grow)   |
    |          |           |
    |          v           |
    |                      |
    |     // free memory   |
    |                      |
    |                      |
    |----------------------| <--------------------
    |  myprogram.o         |                    |
    |----------------------|                    |
    |  mylib.o             |                    |
    |----------------------|              executable image      
    |  myutil.o            |                    |     
    |----------------------|                    |
    |  library code (libc) |                    |
    |----------------------| <--------------------                
    |                      | 0x8000 0000 (=2GB) 
    |                      |
    |  other memory        |
    |                      |
  PAGE_OFFSET =  0xC8000000 = 3GB

  For a 512MB system, the virtual address for kernel will be from   3GB ~ PAGE_OFFSET + 512MB


  1. user space - ( 0x0... 0xbfffffff ), kernel - (0xc0000000 .. 0xffffffff ), swap please

    1. the diagram says the same buddy, I just wrote that it is possible the other way also..:)

  2. it is generally the other way in linux :D

    1. Thats wrong. Kernel starts at 0xc000000. You have it wrong in your article.

  3. Linux splits memory between user and kernel space using PAGE_OFFSET variable. Any memory above PAGE_OFFSET is kernel space and any memory below PAGE_OFFSET is user space. PAGE_OFFSET is architecture dependent and on x86 the value is 0xC0000000

  4. I have one doubt when the RAM size is only 512MB: In this case if I am running a program which doesn't use any system calls, which in tern doesn't shift privilage from user space to kernel space, then how the memory will be allocated i.e. mapping from virtual space to physical space happens? as 512MB directly mapped to kernel virtual address.

  5. copy paste from some other link. original is here http://users.nccs.gov/~fwang2/linux/lk_addressing.txt

  6. What will be the Kernel Space start and end address in 64-bit machines?