[aklug] Re: [aklug] Re: A developer's take on system resource consumption

From: Christopher Howard <christopher.howard@frigidcode.com>
Date: Mon Jun 13 2011 - 15:35:01 AKDT

I wasn't trying to say ur objections were purely philosophical. I was just trying to steer the conversation in a direction that was a little more helpful. When it comes down to it, i think we all agree that memory optimization is important. But what most serious problems that need to be addressed? E.g., in Firefox's case, what would u say are the biggest issues as far as memory consumption? Actual memory leaks? Caching pages too long? Too much functionality? Someone mentioned more efficient data structures, but how far can u really go with that? (E.g., a jpeg in memory really can't be compressed any more)
Sent from my HTC

----- Reply message -----
From: "Arthur Corliss" <acorliss@nevaeh-linux.org>
Date: Mon, Jun 13, 2011 11:49 am
Subject: [aklug] Re: A developer's take on system resource consumption
To: "Christopher Howard" <christopher.howard@frigidcode.com>
Cc: <aklug@aklug.org>

On Sun, 12 Jun 2011, Christopher Howard wrote:

> Interesting discussion. Maybe, though, we should step away from
> philosophy and focus on specifics. To be honest, I'm not qualified
> address the issue of memory management in C/C++ and similar languages,
> because I don't (and won't) program in them anymore. I'm getting pretty
> strongly hooked on Haskell now, and it is quite a different bear. In
> Haskell, at least, you don't allocate memory or assign variables, per
> se: rather, you focus on writing functions that need to "remember" as
> little of the data/calculation as possible in order to return a result,
> and then you tweak the compiler options to produce the best code. Space
> leaks are a serious issue, but are (from what I've read) usually common
> technical mistakes that have been well addressed by the community.
> Here's a link with some examples, if you are interested:
>
> http://blog.ezyang.com/2011/05/space-leak-zoo/

I'm definitely not Haskell-fluent, so can't comment on how they avoid a lot
of the pitfalls. Your link definitely went through some lengths to define
as many types of leaks as possible.

That said, you assert that my objections and arguments are purely
philosophical, and to that I would have to strong disagree. I gave you a
very specific and practical issue to consider in cache coherency. Good
cache coherency is actually somewhat easy to maintain (especially with the
size of today's L1/L2/L3 caches) on a single threaded system with a little
bit of dilligence. On a multitasking system, however, it's a whole
different ball game. And, if any of have looked at the process lists of
your normal working environment, you'll note that *concurrency* is the name
of the game. Working in a contested environment where context switches are
frequent and run queues are always occupied should be the base expectation.

And that basic acknowledgement doesn't seem to be prevalent. Developers
focus (understandably) on their desired functionality to exclusion of all
else. What happens when they achieve their goal, though? The standard
seems to be "optimizing" for performance by finding ways to speed things up
on a *single* threaded system. We see silly compiler options like loop
unrolling and other optimizations applied with no real concern for memory
utilization, processor cache coherency, or anything.

And if you think the problem is still only a minor consideration, consider
virtualization. Environments where your context switches isn't just various
application processes, but entire operating systems. The kernel developers
have at least given this some though, which is why Kernel Samepage Merging
was introduce to support KVM. Ironically, to take advantage of KSM you
still need to run yet another daemon process, and concurrency rises a little
bit more.

So, the kernel guys are trying to help, but what about userland? Sure, some
of the software out there is trying to mark pages as mergeable, but wouldn't
it also be nice if most of the software, by default, tried to lighten the
load, IOW, the number of pages that need to be scanned and/or merged
altogether?

> It is a bit speculative, I suppose, but I think if someone ever got
> around to writing a new Web browser in a functional language, it would
> be the coolest and most powerful browser ever written. Wouldn't mind
> jumping on that project myself, if there wasn't /so much/ involved
> (changing Web standards, malformed HTML, DOM, javascript integration, et
> cetera et cetera) But maybe I will someday.

Web stuff gives me hives. Good luck if you ever tackle that bear.

         --Arthur Corliss
           Live Free or Die

---------
To unsubscribe, send email to <aklug-request@aklug.org>
with 'unsubscribe' in the message body.
Received on Mon Jun 13 15:34:56 2011

This archive was generated by hypermail 2.1.8 : Mon Jun 13 2011 - 15:34:56 AKDT