My previous response was meant to be qualitative rather than quantitative, indicative rather than exhaustive.
I don't currently have access to FreeBSD sources on a local system so it would be quite time consuming to do the necessary trolling through the source files to get detailed answers to your questions.
Here's a bit of quantitative speculation to help illustrate why a fair bit of memory might be needed to "manage" 2GB. A common thing an operating system wants to do is allocate and free pages. Pages can be in a number of different states: free, locked in memory (can't be swapped or paged, example: kernel code, kernel data, active i/o buffers), swappable/pageable etc. We might want to keep some flags about pages (example: a page is shared by a number of users, shared data or code) and a reference count (of sharers) to avoid "freeing" a page which has been released but is still in use by other sharers. So, for each page we might want a data structure with a couple of pointers (to allow quick insertion and removal from doubly linked lists of free pages, locked pages etc, (2 x 4 bytes), flags (say 32 bits), reference counter (say 32 bits). Thats 16 bytes. In a kernel for multiprocessor systems we might also want to add a lock to prevent multiple CPUs trying to change characteristics of the same page at the same time. We might also want to add some debugging information, say 32 bytes all up.
On a system with 2GB and a page size of 4kB there are 512K pages. So an array of page structures (one element per page) will occupy 325121024 bytes = 1610241024 bytes (16MB).
Suppose the kernel is allowed to grow to 1GB and it is decided to preallocate the page map pages for the kernel to avoid the complication of having an attempt to grow the kernel fail. The preallocated page tables require a little more than 1MB of RAM. (Note this is not the same as preallocating the 1GB kernel space.)
@houms:
Thank you for your response… though I am not sure I understand all of it :). If I understand it correctly, your saying the kernel is allowed to preallocated page tables for its own use?
Yes.
something like caching?
yes (f preallocation is like caching)
Is 500 MB reasonable in that regard?
Don't know. I haven't done an exhaustive calculation.
i mean if BIOS reports say 2014MB, then this is real memory, but is it supposed to use +500MB to manage 2014MB?
See previous answer.
and if so, then is it really only managing the avail memory?
My list of examples of the likely use of that 500MB memory was not meant to be an exhaustive list.
I am just trying to understand if it is actually using all the ram? I know that question is a bit of an oversimplification, but is that "unavailable" ram being utilized as ram or for kernel functions?
If FreeBSD swaps or pages its probably using all the available RAM. If it is not swapping and not paging its probably not using all the available RAM.
The default FreeBSD kernel build in FreeBSD 6.x allowed for a maximum kernel size of 1GB. (I don't know about pfSense nor about FreeBSD 7.x.) If the same applied to pfSense in basic firewall applications there would be little point in having much more than 1GB RAM because the basic firewall application generally does most of its work in the kernel. Of course some applications run sometimes so it would be good to have a bit more than 1GB. In a lot of cases, 2GB will be the next practical step up from 1GB. If the firewall runs applications that use a lot of RAM (snort is reportedly one of those) then additional RAM could be useful.
