Cool Mac Gear

I received the following comments on OS X memory management issues from Simone Manganelli, developer of the shareware Memory Usage Getter utility that I reported on here last week.

Mr. Moore --

I just read your article about memory usage on AppleLinks and your quest to find what was eating up all your memory with Memory Usage Getter. As the creator of Memory Usage Getter, I just wanted to shed some more light on the issue that you are having.

Mac OS X indeed does have some great memory management features, far better than OS 9, as you noted in the excerpts from the help files. But the fact remains that Mac OS X loves as much memory as it can get, and it uses all of the available RAM very agressively.

In one of the FAQ's (near the top) in the help files, there's a question about why the total memory usage is much higher than the sum of each individual application's memory usage. In brief (you can read the FAQ there, too), the discrepancy is inactive memory. And inactive memory is basically what your issue comes down to. As you noted, quitting applications and starting them up again didn't really help alleviate your memory usage; a direct consequence of Mac OS X's aggressive memory usage policy.

Here's the thing: when you have free RAM, Mac OS X likes to house data that can potentially be used in the near future. For example, you've probably noticed that when you quit an application and restart it, it takes less time to start up the second time than it did the very first time. This is because the resources needed to launch that application were stored in memory (since you have enough free), and therefore they were indeed needed again when you launched the program the second time. When you open and close many applications, this inactive memory usage can add up. And that's why you're often "hitting the roof" with RAM usage.

In Memory Usage Getter, I've probably misled many people by including the "inactive" memory into the "used" total. Technically, this inactive memory really does have data in it. Practically, though, it's not really being used, so it probably shouldn't be counted. You can actually see the exact value of this inactive memory in Memory Usage Getter (a hidden feature) by clicking on the overall progress bar a few times. It will cycle through wired, active, inactive, free, and used memory usage, where used is wired+active+inactive. A brief description of each of these kinds of memory is given in an FAQ in the help files.

As you noted, though, inactive memory will eventually be paged out once all of your RAM is filled up... this is inevitable with Mac OS X's aggressive policy. However, the speed boost that you gain from having inactive memory far outweighs the cost of eventually having to page out this RAM, because, obviously, you're going to have to page it out anyway -- so why not be lazy about it, in case it needs to be used again? That's how Mac OS X works.

Does this affect stability of applications? Absolutely not. This will only affect the speed of applications, because data in RAM needs to be paged out to disk, and then new data needs to be paged in. The most likely causes of your more frequent crashes in OS X are bugs in the software, and perhaps bugs in the frameworks of OS X (OS X is still relatively immature). All too often I find that it's that quick and dirty ported application that works in OS X, but doesn't work beautifully.

Let me know if this helps to clear up the issue. I'll be happy to at least attempt to answer any more questions you might have.

-- Simone Manganelli
Ellipsis Productions

Thanks Simone -- that clears up more issues. I'm beginning to conclude that OS X is probably functioning normally on my computers, and that my expectations are more than it's up to.

Here is the passage from the MUG Help FAQ referred to above

"I didn't believe the overall memory usage progress bar, and went through the tedious task of adding up the memory usage of all my applications. My suspicions were correct! The cumulative amount does not match the amount displayed on the progress bar. In fact, the progress bar is drastically overestimating the memory usage. Is this a bug?

Allow me to explain.

If you read up on the other sections of the documentation, you'd know that Mac OS X has a very efficient memory managament system. Well there's one thing that I neglected to mention: data often resides in memory even if it is not being used by any application. This is because this memory may be used in the near future, and Mac OS X conveniently stores this data in memory so that it doesn't have to take the time to find it again when it needs this memory. One kind of data that is often stored in memory is resources that are needed to launch applications. If you've ever noticed that applications launch faster after already having launched them once, then you're noticing this feature of Mac OS X.

The problem is, there's no specific process that is using this memory. Therefore, where is it supposed to be reported? Mac OS X is lazy, and doesn't report it anywhere, except when it reports overall memory usage. Much of this stored data is in the form of "inactive" memory, and appears as "used" memory, even though technically it isn't being used. So the discrepancy you are seeing corresponds to this inactive memory.

Memory Usage Getter can be downloaded here:
http://homepage.mac.com/simx/main_page.html

Virtual Memory swapfiles

From G-man

Charles,

I'm finding your treatise on RAM and pageouts quite indulgent. You're overly preoccupied with the difference with OS9 management of memory but there is a blind spot in your argument, OS9 memory management sucked when an application crashed. Period.

Your performance "hit" is a perceptual problem associated with writes to the disk and nothing more. Out of interest, go to "Finder" and select "Go", then "Go to Folder". Type in /var/vm/ you'll see exactly how many swapfiles are in your virtual memory. If you're generating 3-4 after a days work, opening those apps you've mentioned, then you do have a problem.

More importantly, if the pageout value (in brackets) in your "top" output in Terminal is always above zero then you're in real trouble. The normal pageout value doesn't matter all that much, but the value in brackets indicates the number of pageouts happening every few seconds (I think it's a minute). This is very bad if it stays above zero and will give you a problem.

Make your terminal transparent (via window manager in the menu) and use the:

top -u -s4 5

command, keep it open in the background and monitor what's happening much better than you would with any GUI app. Type "man top" in Terminal for more information. I'm not trying to be condescending, we now have the tools to fully understand what's happening at the heart of OSX. OSX performance is slower than OS9, especially in G3 territory because the G4 extensions do help performance with OSX more so than OS9. There is a trade off, some people actually claim that OSX is about as stable as or less so than OS9. Sticks in the mud? It took 4-6 yrs to optimize your Classic use and you can't expect one year of OSX to be that straight forward.

For an interesting and more lucid commentary on this try here.

Please read this site on Classic, it will help a great deal:

G

Howdy G;

Thanks for the advice. I don't pretend to be proficient or knowledgable in the UXIX/command line end of things, and the Terminal is more or less Terra Incognita for me, but I'll try your suggestions.

As for my "treatise," I'm just trying to get a handle on what's going on in terms that a tech-ignoramus like myself can get his mind around.

Yes, crashes suck, but I've never been plagued with many in OS 9. I don't dispute OS X's superiority for users who do/did experience a lot of crashing, but it's not a major issue for me in the old OS. As for OS 9 memory management, for all its acknowledged shortcomings technically, it sems to suit my mode of computer use better than OS X. I was booted into OS 9.1 for a while over the weekend, with all of my usual applications open, and I still had some 440 MB of free RAM available, was enjoying excellent system stability, and a lot more speed than I get in OS X. From a pure performance standpoint, OS 9.1 does a better job *for me* than OS X does. That's a real-world use, non-theoretical, non-technical observation.

OTOH, I'm getting addicted to some applications or application versions that are only available in OS X, and am now using it as my main production system, despite my continued frustration with certain aspects. Speed is actually not too bad even on the 500 MHz G3 after a recent reboot, but it goes downhill as the free RAM gets used up (and it's not just me and my machine -- see Andrew's letter immediately below). System stability is great, but application stability sometimes not so great. I'm losing a lot more unsaved data to unexpected program guits in OS X than I ever have in OS 9.

Charles

Dock, memory leaks

From Andrew Main

Charles,

"Your Dock usage pretty much mirrors mine - ie; it's an application switcher -- just not as good a one as the real AS palette in OS 9."

How is it "not as good"? It does everything the AS does in 9 (shows open apps, and you can drag docs to it to open them) and more, since you can also put favorite app icons, as well as folders and docs, in it for easy access. Though I keep these last to a minimum, I do have a few, those I use all the time--just as I had aliases for them on the desktop in 9.

Thanks for your continuing report on memory issues. I've started using Memory Stick, and am appalled to find that with only half a dozen apps or so running (Eudora, Chimera/Navigator, iCab, URL Mgr, TextEdit, Preview--haven't even opened AppleWorks, my other daily tool), I have only 13% free out of 512MB. I find that obscene. This morning when I awoke my PB after several days running, it was glacially slow. Logging out and in didn't help, but Restarting did. However, after running all day with the apps listed above, memory use has crept up again.

I like a lot about OS X, but I don't much like how poorly it seems to be designed under the slick interface. It doesn't really do <i>that</i> much more than OS 9, that it should be such a glutton for memory. The Baron Harkonnen of operating systems.

Andrew

Hi Andrew;

Thanks to Simone Manganelli and a coterie of tech savvy readers I'm getting more educated on thefiner points of OS X memory management.

As for the Dock's shortcomings vis a vis the Applecation switcher, here are three that come to mind:
- spatial orientation changes
- icons harder to distinguish
- cluttered with multiple functions

Of course some might consider the latter an advantage.

Charles

Color It!, and WannaBe Mac OS X Native Alternatives

From Hi A.

Charles,

On http://www.applelinks.com/articles/2003/02/20030228134055.shtml you mentioned you are using ColorIt and WannaBe in Classic.

For ColorIt I'd recommend using GraphicConverter! http://www.lemkesoft.com/ or Appleworks!

For WannaBe try: http://buddha.kicks-ass.net/macosx/ports.html (Links)

Sincerely,
anonymous

Hi A.

Thanks. I've tried Graphics Converter. It's an excellent and very capable application, but I find it clumsier and slower to use that Color It!, which I've used for years (the thing runs on 68020 Macs!) and love. Appleworks is just too big and clumsy for my needs, and doesn't have enough image editing power. I'm eagerly awaiting the promised OS X port of Color It!

Same deal with Links. Downloaded it back in the fall of 2001, when I was still trying to go all OS X native. Alas, it's simply not as nice to use as WannaBe, which is slick, lightning fast, user-friendly, and has the wonderful facility of a convenient menu shunt to a list of full-featured browsers, which I use extensively (alas -- no Safari or Chimera support yet). I understand that David Pierson is pondering an OS X port of WannaBe.

Until then, I'm inclined to stick with Classic Mode. I'm not being gratuitously perverse. The Classic apps. are just faster and better for the stuff I do. I don't need any more performance slowdowns.

Charles

FTP software for OS X

From Scott R. Godin

"I've never used Interarchy, which I'm sure is an excellent app., but a bit pricy for my purposes. As Kirk noted is his review, he'll use freeware by preference if it'll do the job for him. That's pretty much my philosophy as well. The freeware version of Vicom FTP Client for Classic has served me admirably for my not very heavy duty, but frequent, FTP needs for several years. RBrowser Light is a good freeware FTP client for OS X and there are others."

You've NEVER tried Interarchy (or its predecessor, Anarchie) ? You do *not* know what you've been missing. Yes it's a little pricy, but IMNERHO[1] well worth the price for what it does in addition to simply transferring files.

I've tried a lot of shareware, and this is one of the packages I've gone ahead, kept, and registered. Why? Because software this good needs to be supported and encouraged.

Download it and give it a 30-day trial. Whether or not you decide it's worth the shareware fee is pretty much up to you, but I do guarantee you'll be impressed.

Why? Because unlike most other FTP packages, it works like you would expect a Macintosh app to work. :-) There's all sorts of subtle Mac-like things built in. (as well as functionality I've rarely seen in other FTP apps).

Please don't just pass on it because you think the price is high. Shareware worth its weight in shekels is rare enough as it is. Nothing impressive is completely free. TANSTAAFL.[2] ;)

--Scott
[1]In My Never Even Remotely Humble Opinion ;)
[2]There Ain't No Such Thing As A Free Lunch[3]
[3]Heinlein, "The Cat Who Walks Through Walls"

Hi Scott;

Well, with RBrowser and several other excellent freeware choices in OS X native FTP software, there's really no reason to resort to the CL, unless that's what turns your crank. ;-)

Ans at least with shareware, price is no reason not to check it out. I remember Anarchie being around in the mid-'90s, and i think I even had a copy or two in verious CD bundles, but I had no Internet access then, so never bothered with it.

When I did have call for a bit of FTP work, Fetch was conveniently around, and later Vicom's client suited me very well.

I'll put Interarchy on my check out to-do list.

Charles

The OS X Odyssey archives may be accessed here:
http://www.applelinks.com/news/odyssey/

CM

CM

Charles W. Moore

Cool Mac Gear