[wplug] FW: Windows and Linux Memory Limitation based on application supp
ort of PAE
Wise, Jeremey
jeremey.wise at agilysys.com
Mon Dec 22 14:44:51 EST 2003
Below is the final message I sent to our internal consultants group and have
gotten a lot of 'oh.....REALY???!?!?!?!?' replies back.
Thought it would be of interest to others.
JW
After speaking with the IBM Lotus Notes team, and doing some follow on
research, I have uncovered some interesting facts about how Microsoft and
Linux interact in an IA32 environment where memory above 4GB is required.
The question is "how to applications or operating systems support Intel's
PAE". This is important in that many designs we are involved consist of
applications which are 'non-PAE aware'. This requires that they share the
4GB memory space. PAE memory mapping also requires approximately 15%
(Average based on various numbers in articles)performance overhead due to
memory mapping.
***************************************************************************
***************************************************************************
Windows OS Documentation:
http://support.microsoft.com/default.aspx?kbid=283037
<http://support.microsoft.com/default.aspx?kbid=283037>
Windows has limitations inherent to IA32 that limit ALL versions of Windows
unless the APPLICATION is compiled to support the PAE/AWE API.
"AWE is a set of application programming interfaces (APIs) to the memory
manager functions that enables programs to address more memory than the 4 GB
that is available through standard 32-bit addressing."
http://download.microsoft.com/download/7/5/7/757a5c5c-1ad2-4774-9ffa-ec78052
c42fb/scaling.doc
<http://download.microsoft.com/download/7/5/7/757a5c5c-1ad2-4774-9ffa-ec7805
2c42fb/scaling.doc>
Windows 2003 has added many improvements to the memory manager and total RAM
support but is still limited by the IA32 limitations.
"Support increased from 32 GB on Windows 2000 to 64 GB on Windows Server
2003 for x86-based computers."
"The answer is to take advantage of Physical Address Extensions (PAE) or
Address Windowing Extensions (AWE)(These two terms are used interchangeably,
..... AWE CANNOT be used on the following Operating Systems:
* Windows 2000 Server (Standard)
* Windows 2000 Professional
* Windows XP Home Edition
* Windows XP Professional
* Windows 2003 Standard Edition
* Windows 2003 Web Edition"
Applications supporting PAE Extensions for Windows
Lotus ->No
Exchange ->No http://support.microsoft.com/default.aspx?scid=kb;en-us;827281
<http://support.microsoft.com/default.aspx?scid=kb;en-us;827281>
Microsoft SQL ->Yes
Sybase -> No http://www.dbforums.com/t934449.html
<http://www.dbforums.com/t934449.html>
IBM DB2 -> Yes but NOT versions before 7.2
http://sp81.msi.umn.edu:999/DB2/db2q0/frame3.htm#buffpool
<http://sp81.msi.umn.edu:999/DB2/db2q0/frame3.htm#buffpool>
Oracle -> Yes but ONLY Certain versions
SAP -> Yes but NOT versions before ??
https://microsoft-sap.com/docs/Upgrading_SAP_on_Microsoft_FINAL%20090203.ppt
<https://microsoft-sap.com/docs/Upgrading_SAP_on_Microsoft_FINAL%20090203.pp
t>
***************************************************************************
***************************************************************************
Linux Documentation:
email from RH Kernel Team Fri 12/19/2003:
Kernel memory pages are allocated against a thread and are abstracted from
the application layer, unlike Windows.
"
The information I was able to retrieve regarding Linux memory modeling and
application memory usage on IA-32 follows:
* Recent kernels in the 2.4 series support PAE through
selection/activation of compile options.
* Kernel 2.4.21 in Red Hat Enterprise Linux 3 is delivered in
pre-compiled, binary RPMs with the following options
* Kernel: Uniprocessor, highmem disabled, PAE disabled,
4GB/4GB split disabled,
* Kernel-SMP: multiprocessor (incl. hyperthreading) support,
highmem ENABLED, PAE ENABLED (supports >4GB physical RAM,
4GB/4GB split disabled,
* Kernel-hugemem: multiprocessor support, highmem ENABLED,
PAE ENABLED, 4GB/4GB split ENABLED (for systems w/ >= 16GB
physical RAM
* Applications access all memory presented by highmem/PAE/4GB
split in a "generic" fashion using standard malloc() calls.
Each application usually spawns several processes - recall
that 4GB split offers up to ~3.7GB memory to each process,
resulting in much more "head room" for processes.
* Applications typically do not need specific coding to address
memory in the "PAE range." The application(s) may need to be
recompiled in order to use all memory presented by the kernel
(i.e. use the full ~3.7GB process space).
"
http://www.spack.org/index.cgi/LinuxRamLimits
<http://www.spack.org/index.cgi/LinuxRamLimits>
Though this is a limitation of ALL IA32 architectures this just documents
the fact on Linux platforms. No SINGLE THREAD can address more than 3GB.
"PAE does not increase Linux's ability for *single* processes to see greater
then 3GB of RAM..."
" With 2.4 kernels (with a large memory configuration) a single process can
address up to the total amount of RAM in the machine minus 1GB(reserved for
the kernel), to a maximum 3GB."
http://www.ussg.iu.edu/hypermail/linux/kernel/0105.1/1169.html
<http://www.ussg.iu.edu/hypermail/linux/kernel/0105.1/1169.html>
As we see here. The kernel 2.4ver is limited by the fact it has a limit in
the 'remapping' space for doing page references up to the RAM beyond 4GB.
"As Linux allocates 3GB of this address space to user
+ space, and the kernel reserves 128MB of address space for its needs,
+ only 896MB of address space is available for mapping physical
+ memory."
http://kerneltrap.org/node/view/620 <http://kerneltrap.org/node/view/620>
This refers to similar enhancements made to memory addressing (similar to
those referenced above for Windows kernel) in eliminating older PAE/AWE
inefficiencies by building better memory cluster maps.
"...page clustering, which by reducing the number of pieces of memory the
kernel keeps track of shrinks the array of pages to fit nicely into a 1GB
kernel virtualspace without taking anything away from userspace..."
http://www.prism.gatech.edu/~gte213x/LinuxMM/rpt.html
<http://www.prism.gatech.edu/~gte213x/LinuxMM/rpt.html>
It appears that backwards compatibility support for PAE/AWE support has been
added.
"By 2.2.17 (kernel)... If the CONFIG_HIGHMEM_64G option is selected, then
PAE is turned on and the kernel can access up to 64GB of RAM. This support
must be compiled into the kernel [CONF]. However, even though your machine
may support greater than 4GB of memory, no user process can use more than
4GB of memory at a time (again, the 32-bit pointer limit)."
http://www.ussg.iu.edu/hypermail/linux/kernel/0105.1/1169.html
<http://www.ussg.iu.edu/hypermail/linux/kernel/0105.1/1169.html>
Additional references to maximum memory support per SINGLE thread to 3GB.
Same as Windows.
"Configure. help in 2.4.5-pre1 (kernel).... This will result in a "3GB/1GB"
- split: 3GB are mapped so that each process sees a 3GB virtual memory
- space and the remaining part of the 4GB virtual memory space is used
- by the kernel to permanently map as much physical memory as
- possible."
Real world comments:
http://www.linuxmanagers.org/pipermail/linuxmanagers/2003-May/001176.html
***************************************************************************
***************************************************************************
VMWare Documentation:
http://www.vmware.com/support/gsx/ts/host_suse717273_gsx_linux.html
" SuSE Linux 7.1, 7.2 and 7.3 SMP kernels implement 64GB PAE memory and are
not supported. For SuSE 7.1 and 7.2 hosts, the non-PAE-enabled versions of
these kernels can be installed from the RPM package available on the SuSE
distribution CD-ROM (/suse/images/k_psmp.rpm). For a SuSE 7.3 host, you can
select the non-PAE-enabled kernel by running the YAST2 configuration
utility."
http://www.vmware.com/support/gsx/doc/sysreqs_gsx_linux.html
<http://www.vmware.com/support/gsx/doc/sysreqs_gsx_linux.html>
Their appears to be some question as to whether VMS are being emulated back
a Pentium Pro chip (or better) when a host OS is running under a VM.
" You need enough memory to run the Linux host operating system plus memory
required for each virtual machine's guest operating system and applications
on the guest and host. ...Minimum: 256MB (512MB recommended) ...Maximum: 4GB
(for kernels in the 2.4.x series; otherwise 2GB)"
These are quotes in conversation with a VMWare SE.
"GSX can address up to 64GB of RAM but the VM can not support more that 2GB
and 1.5GB per VM (new beta releases are looking to move that to be the same
as ESX 3.6)"
"VMs can only see a maximum of 3.6GB of RAM. This is due to the 4GB limit
plus a little bit of mapping overhead."
"Even where, like Opterton, the native OS can run 64bit the VMs would be
still a 32bit instance and have the limitation of 4GB."
"We do not emulate the CPU back to the VM ... the underlying VMWare OS is
just a 'traffic cop' to the native hardware. ... we do nothing to the
instructions."
Applications supporting PAE Extensions for Linux
Lotus -> NA
Exchange -> NA
Microsoft SQL -> NA
Sybase -> NA
IBM DB2 -> NA
Oracle -> NA
SAP -> NA
***************************************************************************
***************************************************************************
Summary:
There are two ways to architect an application. Scale up (big iron) and
scale out large numbers of commodity hardware). Correct architecting or
scale up applications require an understanding of whether the can support
the hardware architecture. Windows applications require that they support
the AWE API. If they do not the application is relegated to a single 3GB
memory space. This memory space is shared amongst all 'non-AWE'
applications. Many applications have been recompiled to support AWE but do
not assume this to be the true, you may be architecting a non-supported,
non-scalable solution.
Linux treats RAM differently in that the applications are completely
abstracted from the memory subsystem. They do not need to be recompiled to
support PAE functions so long as the Kernel itself can address the RAM. This
alleviates some of concerns for architecture designs but application support
is still kernel version dependant.
VMWare's Operating System is based on Linux and so fills out that matrix of
support in addressing RAM at the hardware level but does NOT map this
transparently to the VMs. A "Virtual Machine" is emulated back the hardware
on which it runs but without any AWE/PAE functionality. This means that the
host VM is limited not only by the choice of OS but also by the fact that it
does not believe it is being "installed" on hardware supporting AWE.
IA32, irregardless of the Operating System, will never be able to allocate
more than 4GB of RAM for a single thread. Most applications spawn hundreds
or thousands of threads and so are rarely impeded by this limitation.
IA64 (including AMD 64) eliminates all of these issues. So long as the
application is supported on IA64 there are no limitations to the memory
addressability (until we reach Petabytes of RAM).
Jeremey Wise (440)-519-6006
Systems Consultant(CNE,MCSE,CSE)
Agilysys, Inc.
Jeremey.Wise at Agilysys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://penguin.wplug.org/pipermail/wplug/attachments/20031222/70e2e331/attachment-0001.html
More information about the wplug
mailing list