Mark Stosberg

Running Perl tests in parallel with "prove", with some exceptions

2012-08-28T01:18:27Z

Perl’s prove is a great tool for running TAP-based test suites. I use it for a large project with about 20,000 tests. Our test suite runs multiple times per day. At that scale and frequency, the run time is a concern.

To address that, prove has -j option, allowing you to run tests in parallel. Unfortunately, in our large test suite not all tests are “parallel ready”. I needed a way to run most tests in parallel, but mark a few to be run in sequence. Ideally, I’d like these tests to be part of the same run as the parallel tests.

After much digging around, I found that prove has had support for adding exceptions to parallel tests runs since in 2008. The feature has just never been documented in prove, making it difficult to discover. Here’s an example which works with the currently released version of prove, 3.25:

# All tests are allowed to run in parallel, except those starting with "p"
--rules='seq=t/p*.t' --rules='par=**'

For more details and documentation, you can see my related pull request. You can leave a comment there to advocate for or against merging the the patch, or you can download the raw patch directly to apply to your own copy of the Test-Harness-3.25 distribution.

*UPDATE 9/14/2012: * It turns out it doesn’t work quite as I thought. I tried tweaking some of the internals of prove further, but hit a bug. See the details in my follow-up to the Perl-QA list. I changed approached and started working on making my exceptional tests parallel-safe. Perhaps I can cover some of the techniques I used for that in a future post.

Plack::Request: not as read-only as it might seem

2011-02-05T14:24:16Z

For me, one of the promises of PSGI and Plack is to get away from programming with global variables in Perl, particularly being able to modify the request object after it’s been created. Long ago, CGI.pm replaced using a global

%FORM

hash with a “query object”, but it essentially has been used as always-accessible read/write data structure.

It would appear at first look that improving this might a fundamental part of the new system: A lexical environment hashref is explicitly passed around, as seen here:

Plack::Request->new($psgi_env);

vs the old implicit grab from the environment:

CGI->new;

You might hope to be escaping some of th action-at-a-distance badness of global variables, like having a change to environment alter your object after the object is created. You might like avoid having changes made to your object avoid changing the global environment as well.

Not only does Plack::Request require an explicit environment hash to be passed in, both nearly all methods are read-only, including a param() method inspired by a read/write method from CGI.pm of the same name. That’s all good.

This would all seem to speak to safety and simplicity for using Plack::Request, but the reality turns out to be far muddier than you might hope. I encourage you to down and run this short, safe interactive perl script which illustrates some differences. It shows that:

Plack::Request objects can be altered after they are created by changing the external environment.
Modifying a Plack::Request object can potentially alter the external environment hash (Something which CGI.pm explicitly does not allow).

In effect, the situation with global variables is in some regards worse. Plack::Request provides the impression that there is a move away from action-at-distance programming, but the fundamental properties of being affected by global changes and locally creating them are still present.

On the topic of surprising read/write behavior in Plack::Request, you may also interested to note the behavior of query_parameters(), body_parameters() and parameters() is not consistent in this regard. I submitted tests and suggestion to clarify this, although that contribution has not yet been accepted.

Here’s the deal: The hashref returned by query_parameters() and body_parameters() and parameters() are all read/write — subsequent calls to the same method return the modified hashref.

However, modifying the hashes returned by body_parameters() or query_paremeters() does not modify the hashref returned by parameters(), which claims to be a merger of the two.

It seems that either all the return values should be read-only, ( always returning the same values ), or if modifying them is supported then the parameters() hash should be updated when either of the body_parameters() or query_parameters() hashes are updated.

Reflections

An incoming HTTP request to your server is by it’s nature read-only. It’s analogous to a paper letter being delivered to you be postal mail.

It’s a perfect application for an immutable object object design that Yuval Kogman eloquently advocates for. Plack::Request comes close to implementing the idea with mostly read-only accessors, but falls short. The gap it leave unfortunately carries forward some possibilities for action-at-distance cases that have been been sources of bugs in the past. I’d like to see Plack::Request, or some alternative to it, with the holes plugged: It should copy the input, not modify it by reference, and parameter related methods should also return copies rather than reference to internal data structures.

Best practice for handling UTF-8 when percent-encoding?

2010-12-20T03:47:29Z

In my previous post I summarized the current state of Percent-encoding in Perl. One of my conclusions was that the perfect percent-encoding solution would automatically handle UTF-8 encoding, using logic like this:

utf8::encode $string if utf8::is_utf8 $string;

Respected Plack author miyagawa quickly responded in a response post to say that the above code approach is a bug, although the code pattern is already wide use as it is present in Catalyst, CGI.pm (and by extension CGI::Application and other frameworks) as well as Mojo.

In one sense, he’s right. The pattern goes against the advice found in the official perlunifaq documentation which states that

It’s better to pretend that the internal format is some unknown encoding, and that you always have to encode and decode explicitly.

In other words: don’t use the “is_utf8()” function.

Before drawing a conclusion whether this code pattern is the best design in in practice, let’s look some related facts about the matter.

]]>

The W3C is clear that it is a best practice to convert text to UTF-8 before percent encoding it.

However, as Miyagawa points out, only newer protocols, since about 2005, are expected to follow that. The HTTP protocol is far older, and it is valid to pass binary data through HTTP query strings.

But conveniently, Perl reads binary data as 8-bit characters. You can read in /usr/bin/perl, pass it to uri_escape, and it will percent-encode it just fine— without blowing up due to the “hi-bit” characters that some UTF-8 characters correspond to.

In practice, the approach of automatic encoding used by Catalyst and CGI.pm has not been problematic. From reviewing both both bug queues, I don’t see any open or previously-closed bugs that haven caused by the current behavior. It does seem reasonable and logical that it should be safe to UTF-encode text that is marked as UTF-8.

People would reasonably expect the percent-encode/decode process to be a symmetric round trip— you should get back exactly the data that you started with. When UTF-8 encoding is added to the process, this isn’t necessarily the case, since UTF-8 encoding happens only before percent-encoding. The reverse does not happen when percent-decoding. The one way trip to UTF-8 is probably what you wanted, but since the conversion is automatic, the intent of the programmer is not explicit.

The most popular percent-encoding solutions for Perl— CGI.pm and URI::Escape— both pre-date proper Unicode support in Perl, and both include built-in code to re-implement UTF-8 functionality in case you are using Perl 5.6 which lacks proper Unicode support. So in the context of designing those modules, it wasn’t really an option to tell people just call encode($string) if they wanted to UTF-8 encode their text before it was URI-encoded.

So, then, which of these two designs represents a best practice for handling UTF-8 encoding in combination with percent-encoding in Perl?

First option: The popular solution used in practice to UTF-8-encode data if the UTF-8 flag is set. The approach automatically handles the recommended practice of UTF-8 encoding so users don’t have to think about it.
Second option: UTF-8 handling should not built-into a percent-encoding solution. Following the advice of perlunifaq, an ideal solution would not check the UTF-8 flag, but instead would instead offer clear documentation that advises the user about the best practices about possibly UTF-8 encoding text before it is percent-encoded. This design cleanly externalizing all the character encoding issues that sometimes get bundled with percent-encoding, such as translating from EBCDIC or UTF-16 surrogate pairs. It provides users a bit of education about UTF-8 best practices which they may able to apply in other areas. As long as UTF-8 handling is attempted to be automatic, programmers can continue to stay in the dark that the best practice is for them to explicitly handle character encodings themselves. The solution might be as simple as documenting this in the synopsis:

use Encode 'encode_utf8'; $uri = uri_percent_encode(encode_utf8($string));

Note that URI::Escape currently essentially already has the code part of the second alternative in place. It offers percent-encoding without any character encoding, as well as uri_escape_utf8, which is basically just sugar for the code sample above.The URI::Escape documentation explains in precise technical terms what the two alternatives do, but offers very little guidance on which method to choose—the plain one or the UTF-8 one? It doesn’t address which reflects a best practice, or cover the possiblity of any drawbacks to choosing the UTF-8 version. A Google code search shows that there are about 10x more hits for “uri_escape()” vs “uri_escape_utf8()”. I suspect that many of these cases would benefit from being able to handle UTF-8 characters, but the programmers weren’t really educated about when to choose that option. Considering that masses of programmers aren’t suddenly going to become UTF-8 experts, it’s clear to me that an ideal solution should go with one of the option above: Either try to handle UTF-8 automatically, or offer strong guidance in documentation on percent-encoding about best practices regarding character encodings. I think with clear, informative documentation, the second option could be the better way to go.

What do you think?

Percent-encoding URIs in Perl

2010-12-17T16:45:45Z

This post is about the current state of URI encoding in Perl. This is the problem space of being able to safely pass arbitrary text into and out of a URI format. If you’ve even seen a space in URL represented as “%20”, that’s the topic of the moment.

The best general introduction I’ve found on the topic is the Wikipedia page on Percent-encoding.

RFCs on the topic include the 2005 RFC 3986 that defined the generic syntax of URIs. It replaces RFC 1738 from 1994 which defined Uniform Resource Locators (URLs), and RFC 1808 from 1995 which defined Relative Uniform Resource Locators. Sometimes this transformation is called “URI escaping” and sometimes it’s refered to “URL encoding”. RFC 3986 clarified the naming issue:

“In general, the terms “escaped” and “unescaped” have been replaced with “percent-encoded” and “decoded”, respectively, to reduce confusion with other forms of escape mechanisms.”

Elsewhere it’s clarified that percent encoding applies to all URIs, not just URLs.

I think the Perl community would do well to adopt “percent encode URI” and “percent decode URI” as ways to describe this process that is unambigous and in line with the RFC.

There are two URI percent-encoding solutions in Perl that seem to be in the widest use. Both have a significant deficiency.

]]>

Percent-encoding with CGI.pm

The first is CGI::Util which provides escape() and unescape() as a pair. This solution has a lot going for it— it’s been in the core for years, it works back to Perl 5.6, it automatically handles UTF-8 encoding, and it handles some edge cases like EBCIDIC encoding and UTF-16 surrogate pairs. Further you can use escape() and unescape() without using the rest of CGI.pm or ever creating a CGI.pm object. There’s just one major deficiency: These methods have never been documented! Many take advantage of them by using CGI.pm directly or indirectly, as CGI.pm uses them internally. A few people have found them and use them directly. As someone with commit access to the CGI.pm repo, I’ll be documenting them shortly, once I’m done with the detour that became this post.

Percent-encoding with URI::Escape

Probably the most intentionally widely used module for URI percent encoding is URI::Escape. URI::Escape is not in the core, but the URI distribution depends only on MIME::Base64, and that module is not actually needed for the URI::Escape functionality. Like CGI.pm, URI::Escape also advertises support back to Perl 5.6.1. It does not handle EBIDIC or UTF-16 surrogate pairs, but as I’ll explain later, it’s questionable whether those abilities are truly desirable to be built-in to a percent-encoding solution. The deficiency with URI::Escape is that doesn’t handle UTF-8 automatically like most other solutions do. Many perl scripts and modules have called URI::Escape::uri_escape expecting that it will always “just work” for encoding all text. Instead, you have to explictly ask for UTF-8 handling by calling uri_escape_utf8() instead. To credit URI::Escape, it has clearly documented how it behaves in this regard, but it seems like a missed opportunity to handle UTF-8 input automatically. By contrast, most other solutions handle either case automatically with a single line like this:

utf8::encode $_[0] if utf8::is_utf8 $_[0];

RFC 3986 is quite clear that UTF-8 encoding should be part of the solution:

“Non-ASCII characters must first be encoded according to UTF-8 [STD63], and then each octet of the corresponding UTF-8 sequence must be percent- encoded to be represented as URI characters”

URI::Escape is likely suffering from being far older than RFC 3986, and added a new method specific to UTF-8 to keep uri_escape() perfectly backwards compatible. In hindsight from 2010, I think that was an unfortunate choice.

Summary of all known percent-encoding solutions for Perl

I researched further to see what other percent-encoding solutions for Perl and how they differ. Here’s what I found, including CGI.pm and URI::Escape again for completeness.

CGI::Util

Has the benefit of being in the core, but the drawback of being undocumented as of version 3.50.

Names: escape / unescape
Min Perl version: 5.6.0
Handles UTF-8 handling: Yes, on Perl 5.8 and newer
Notes: also handles EBCIDIC and UTF-16 surrogate pairs.

CGI::Simple

CGI::Simple 1.112 Appears to have a bug regarding RFC 2396, section 2.2, concerning reserved characters. It explicitly translates spaces to “+”, unlike most other solutions here which translate it to %20. It also lacks automatic UTF-8 handling. It’s implementation is notably not compatible with the one in CGI.pm, as some would assume.

Names: url_encode / url_decode
Min Perl Version: 5.6.1
Handles UTF-8 encoding: No.
Notes: The implemention here isn’t the same as a second one in the distribution, in CGI::Simple::Util.

CGI::Simple::Util

A second percent-encoding in CGI::Simple 1.112, it is not compatible with CGI.pm’s implementation either. Compare:

CGI::escape                  å -> %C3%A5%20X
URI::Escape::uri_escape_utf8 å -> %C3%A5%20X
CGI::Simple->url_encode      å -> %E5+X
CGI::Simple::Util::escape    å -> %E5%20X

Names: escape / unescape
Min Perl Version: 5.6.1
Handles UTF encoding: No.
Notes: Handles EBCIDIC encoding, inherited from CGI.pm before the fork.

Mojo::Util

Mojo::Util 0.999941 provides a modern, simple implementation with automatic UTF-8 encoding. My gripes with it are that the names say “url” and “escape” instead of “uri” and “encode” to follow the RFCs more closely. It also doesn’t allow you to use a rather normal syntax: Mojo::Util::url_escape('å').. That’s because Mojo has uses the unconventional impementation of modify the input by reference instead of returning a modified copy. Presumably this is done for performance.

Names: url_escape / url_unescape
Min Perl Version: 5.8.7.
Handles UTF-8 encoding: Yes.

Tie::UrlEncoder

Tie::UrlEncoder 0.02 provides a unique interface through a %urlencode hash. However, it doesn’t provide a decoding routine. Basic UTF-8 tests pass for it, but the solution employed is unothorodox. Instead of calling UTF-8 related functions, it calls use bytes;. Official Perl documentation is clearly opinionated this approach. In perlunifaq, it says plainly “Don’t use it.” in regard to use bytes;.

Names: %urlencode
Min Perl version: 5.6.
Handles UTF-8 encoding: The implemention does not follow best practices. See above.

URI::Encode

Not be confused with URI::Escape, URI::Encode is meant to be a newer and simpler take on the problem space. It offers automatic UTF-8 encoding, and includes an option on whether are not to include reserved characters— The option to not encode reserved characters is nice for those who know what they are doing. Unfortunately, it has a poor object-oriented UI. It offers a constructor which does nothing, when the reserved characters option could be used as option there. Then, it doesn’t document that you can call the key methods as class methods to bypass the do-nothing constructor. While it also offers a procedural interface, it’s implemented in terms of calling the do-nothing constructor every time, adding an unnecessary penalty.

Names: uri_encode / uri_decode
Min Perl version: Perl 5.8.1.
Handles UTF-8 encoding: Yes

URI::Escape

URI::Escape provides three APIs, two that don’t handle UTF-8 encoding and one that does. It’s popular, works well and is well documented. It’s main drawback is that UTF-8 encoding is not automatic in uri_escape() and as a result and has not been used by many applications, when UTF-8 support here could have otherwise been a free benefit. uri_escape_utf8() can be used for UTF-8 support.

Names: uri_escape / uri_escape_utf8 / uri_unescape / %escapes
Min Perl version: 5.6.1
Handles UTF-8 encoding: Not automatically

URI::Escape::XS

It sounds like a module that’s compatible with URI::Escape, only faster due to a C-based XS implementation. It does benchmark to be much faster, and it is somewhat compatible, but it lacks a uri_escape_utf8 method, which could be a valuable addition for better compatibility. Instead, it has a uri_escape method that includes UTF-8 support automatically. It also has a higher minimum Perl requirement— Perl 5.8 vs 5.6, which is another important difference that’s not documented. As an additional benfit, Any::URI::Escape exists which will use the XS version if it exists, and the Pure-Perl version otherwise. The wrapper module also unfortunatley glosses over the difference in UTF-8 handling in the XS version and the pure-Perl version.

Names: uri_escape / uri_unescape
Min Perl version: 5.8.1
Handles UTF-8 encoding: Yes
Notes: Requires a C-compiler (but very fast)

Recommmendations

All the URI percent encoding solutions I reviewed had flaws, but the pieces are all there to produce an optimal solution. Here’s my recommendation for designing a perfect solution:

Name the module URI::PercentEncode;
Name the functions uri_percent_encode() and uri_percent_decode()
Return the changed value (don’t modify by reference)
Require at least Perl 5.8.1. Supporting older version is unnecessary baggage at this point.
Don’t build in support for getting data into UTF-8 beyond a simple call to utf8::encode(). Anything else belongs in the domain of the “Encode” module. If I’ve wrong about including support for UTF-16 surrogate pairs in a percent encoding solution, let me know.
Automatically handle UTF-8 encoding (like this: utf8::encode $_[0] if utf8::is_utf8 $_[0];)
Use faster XS-based code by default, but allow building a Pure-Perl version for those who need or want it. (Follow the model of Params::Validate here).

Any volunteers?

That’s my take on URI percent-encoding in Perl. What do you have to add?

Update: *See the reply by miyagawa who states that this code is a bug: utf8::encode $_[0] if utf8::is_utf8 $_[0];. It is used by CGI::Util, Mojo::Util in the versions given above as well as in Catalyst. URI::Escape and URI::Encode do UTF-8 encoding without checking the UTF-8 flag. He has more experience with UTF-8, and I defer to his advice here. *

generating HTTP headers: sorted or unsorted?

2010-01-26T01:42:21Z

Recently I’ve been reviewing how various Perl frameworks and modules generate HTTP headers. After reviewing several approaches, it’s clear that there are two major camps: those which put the response headers in a specific order and those which don’t. Surely one approach or the other would seem like it would be more spec-compliant, but RFC 2616 provides conflicting guidance on this point.

The bottom line is that spec says that “the order in which header fields with differing field names are received is not significant”. But then it goes on to say that it is a “good practice” (and it puts “good practice” in quotes) to order the headers a particular way. So, without strict guidance from the spec about the importance of header ordering, it would be interesting to know if header order caused a problem in practice.

The Plack::Middleware::RearrangeHeaders documentation suggests there is some benefit to strict header ordering: “to work around buggy clients like very old MSIE or broken HTTP proxy servers”

You might wonder what the big deal is— why not just stick to the “good practice” recommendation all the time? The difference can be seen in the benchmarks provided by HTTP::Headers::Fast. By ignoring the good-practice header order, an alternate implementation was able to speed-up header generation to be about twice as fast. Considering that a web-app needs to generate a header on every single request, making header generation smaller and faster is potentially a tangible win, while also still being spec-compliant.

]]> So let’s look at who is doing what:

CGI.pm does not order headers by default or have an option to do so. CGI.pm is used by default by CGI::Application and also by Mason and Jifty when they run under CGI.
CGI::Simple predictably models CGI.pm’s behavior
Dancer generates headers itself, without regard to order
Plack is a mixed bag. If you generate your own headers, I believe they will be passed through unmodified, but if you use Plack::Request they will be ordered (with no option to disable this). With Plack you also have the option to the the RearrangeHeaders Middleware if you want to be certain that your headers are in the “good practice” order.
Mojo generates its own headers, always in the “good practice” order, with no way to disable the ordering.
Catalyst, Plack and others rely on HTTP::Headers to generate headers. HTTP::Headers also currently always generates headers in good-practice order, although there has been some discussion about adding an option to produce the headers in an unsorted order.
And finally, I found HTTP::Headers::Fast which is used by HTTP::Engine and provides the user equal options to generate headers in a sorted or unsorted order.
For a bonus consultation, I looked at Rack, a web server written in Ruby sometimes paired with Rails, and one of the inspirations for Plack. It appears that that Rack does not order HTTP headers either.

So which approach is best? I think what makes most sense to me is leave HTTP headers unsorted by default. This complies with the spec and can be accomplished with a simpler implementation and better performance. The good-practice ordering is a nice-to-have option if you know you need it or want the tradeoff.

And should we be worried about header ordering issues in practice? With CGI.pm in use for over a decade, I can’t recall any header-ordering bugs of the hundreds of CGI.pm bugs I’ve triaged in that bug tracker. I’m interested to know if there are any real-world web clients or proxies that require or benefit from the good-practice ordering.

The cost of saving sent e-mail

2010-01-20T01:17:21Z

I don’t tap my own phone. I don’t xerox postcards before I mail them back from vacation. I don’t take a voice recorder when I go out with friends. And I don’t have a copy machine at home to duplicate hand written notes I may send.

But if I send a message of equal importance by e-mail, then my e-mail program will automatically save a copy of every one of these messages.

E-mails I don’t need waste my time. They increase the time it takes to search and browse through old email. They increase the time it takes for my email to “sync” when I want to go offline. To continue to save every e-mail I send perpetuates the unsustainable myth that as long as our actions are online they are “green”.

]]> First the small action of saving a e-mail is amplified by the 1.5 billion people using e-mail globally. More saved e-mail means more disks to store the e-mail, larger backup systems to handle the volume, and faster processors so that all the archived messaged can be searched through efficiently. Since sent mail accumulates over time, an increasing amount of resources are needed to handle the saved mail. In five years there will be more people on the planet. It’s likely a greater percentage will be sending e-mail, and likely the e-mail volume will be even higher than it as now. What is the energy cost, now and in the future, of storing so much e-mail?

While we can debate the magnitude of the impact, more data storage equals more resource consumption. In the United States, data centers already draw more power than our TV usage [1], and data centers are only part of the energy consumed by our increasingly networked life. In turn, about 40% of our water supply is devoted to generating power. Water is used in part to provide cooling massive data centers. (Only 15% of the water supply is actually used by the public). [2]

If you’re with me on this one, here’s one tip that could make big reduction in the size of your “Sent Mail” folder, while still retaining the memory the correspondence:

Consider not saving a copy of attachments you send in your Sent Mail folder. Attachments are often 10 to a 1000 times larger than a typical e-mail, and you already have a copy of the document on your hard drive. Plus, your recipient is about to receive a copy and she may then also download a copy from her e-mail to her hard drive. Saving the attachment in your Sent folder could mean keeping a forth copy of the document. If it’s important to have a record that the attachment was sent, you could send one message with the correspondence that references the attachment, and save that a message. Then, send the attachment in it’s own message, and don’t save the attachment in the Sent folder.

Another idea: While some places have data retention rules (or laws!), these typically do not apply to personal e-mail accounts. Consider turning off the option to save e-mails by default, and conciously choose which e-mails you think are important enough to save. Note that Google’s Gmail service (146 million users) does not have an option to turn off automatically saving the messages you send. Yahoo is another web-based e-mail provider that does provide this option. Check your e-mail program for details.

It is powerful to ask “what is the impact of this?”, whether you consider e-mail storage, the toxic batteries in our cell phones, or the impact of broadcasting wifi radio waves through our homes 24/7. A daily choice such as a not saving an sent e-mail can be a mindful practice to connect our abstract online lives with the real world.

For details about calculating the carbon footprint of e-mail storage, read on.

Calculating the carbon footprint of e-mail storage

Research about the carbon footprint of e-mail storage turned up little in the way of existing estimates. emailfootprint.org estimated that it would take 1 kilowatt hour to store 1 Gigabyte of data for a year, but they didn’t explain how they came up with that number, and the site is now offline, but accessible through the google cache

TreeHugger referenced a report that said that it would take 1 lb of coal to store 20 megabytes of data for a year, according to the US Department of Energy. But the report they linked to is no longer available. [3]

I decided to see if I could piece together my own estimate of the impact of storing sent e-mail, based on my own usage.

I have about 40 megabytes of mail saved from messages I sent in 2007. This perhaps already represents removing some messages which contained large attachments. For the sake of the example, let’s assume this is an average amount. ( To personalize the example, check the size of your own Sent Mail folders!). Let’s multiply this by an approximate 1.5 billion e-mailers and you get about 56 petabytes of data (about 58 million gigabytes). To picture that: imagine all the data was crammed on to reasonable large hard drives: 500 gigabytes each. It would take about 117,000 hard drives to store that data, assuming it was stored relatively efficiently and no extra space was required to store the operating system on this drives! The actual number is would be higher because much data can be expected to be on older hard drives, manufactured before the very large drives were an option. Further, it’s an industry best practice to use always duplicate data using “RAID”, so the same data would be written to at least two hard drives. Large providers such as Google may further be duplicating data in at least two data centers, for extra redundancy.

Using a number from here, I’ll estimate that it takes 446 MegaJoules of energy to produce a hard drive, which I’ll convert to 124 kilowatt hours. So, it would take about 14.5 million kilowatt hours of energy just to produce 117,000 hard drives, without getting into the energy required to keep them turned on.

Using data from US Department of Energy, it looks like we can expect about 2 lbs of CO2 to be generated to produce each kilowatt hour of energy.

So that puts us at an estimated 29 million lbs of C02 generated to produce enough hard drives to efficiently store all the sent e-mail in the world. (Using my own amount as an average).

To visualize that number, let’s equate it the number of miles you’d have to drive in average car to generate the same amount of CO2. According to Streetsblog the average car generates 1.2 pounds of CO2 per mile, equating to about 24 million miles.

In perspective: 24 million miles is a very large absolute number, but it pales in comparison to the estimated 5 billion miles that American’s drive each day. [4]. The real danger to address is the way of thinking that a digital paperless life is automatically green one.

This post is a follow-up to one entitled Stewardship and Sustainability of our Online Lives.

References:

an EPA study stating that the data center industry devours 61 billion kWh of energy annually compared to ...about 275 million TVs currently in use in the U.S., consuming over 50 billion kWh of energy each year
Quenching the Thirst of Power-Hungry Data Centers, citing primary data from the US government.
The Footprint of Gmail: How Much Energy Would Deleting Email Save?
Americans drive 5 billion miles per day

Modifying PDFs so they open full screen

2009-11-30T02:49:01Z

The PDF spec includes an option to cause PDFs to open full screen when users open them. I’m a fan of the feature because it maximizes screen real estate and creates a simple, focused, experience for the PDF readers. Using this option is one of my two essential tips for creating an impactful newsletter targeted at being read online. The other tip is to use a “portrait” format document, to match the shape of most screens.

Many PDF viewers respond to PDFs that are set to open full screen, but a number of PDF generation tools don’t provide you option to set this preference when creating PDFs. I ran into this with Xournal which is a nice application for Linux-based tablets, but offers no PDF export options.

So I found a way to update a pre-existing PDF to set the preference to have it open full screen by default. The key here is that PDF is a text-based format, so preferences in it can be updated manually by opening and editing the file according to the PDF spec, or the same effect can be accomplished with automated tools. In this case, I found that I needed to update a line that started like this:

<< /Type /Catalog

After /Catalog, this is all that needed to be added:

/PageMode /FullScreen

I automated this with a simple script that I named make-pdf-full-screen.sh. It works for the simple case when no “PageMode” has been declared, as in the Xournal case. I don’t expect it would update the PageMode properly if it was already declared. For a safer solution consider opening the PDF in a text editor to manually set “/PageMode /Fullscreen” on the initial /Catalog line. Alternatively, you could use a formal solution like PDF::API3::Compat::API2 which appears to have the features needed to solve this with Perl.

Here’s the contents of my little script to automate the update:

#!/bin/sh
# usage: make-pdf-full-screen.sh file.pdf
#   The file will be modified in place so that it opens full screen.
#   The current approach is naive... it assumes no Initial View has been defined.
# by Mark Stosberg
perl -pi -e 's?<< /Type /Catalog?<< /Type /Catalog /PageMode /FullScreen?' $1

Stewardship and Sustainability of our online lives

2009-11-26T05:04:09Z

A few weeks ago I had my laptop stolen. Earlier that morning I had been reflecting and writing on the laptop about the intersection of our spiritual lives with our digital lives. And then, as if by divine intervention, my laptop disappeared— during church service no less— and I was given an even greater opportunity to answer the question: When we spent more time browsing the web, what is that we are doing less of? When we spend more checking e-mail, what are we doing less of? And when we spend more time on Facebook, what are we spending less time doing? Apparently, the answer in my case is cleaning is my desk and organizing the garage. Those are the things I did more when I could do the the internet less. I joke about this, but I do envision my home as a place of rest and rejuvenation, yet I let clutter accumulate while I spent more time on my computer doing “productive” things.

There are many implications of shifting our increasingly precious free time online. Today I’d like to delve into the carbon footprint of our online lives.

You can use the audio player here to listen to a 15 minute version of the message delivered at my church, (or you can also download the audio file.)

The message continues below the jump.

]]>

As individuals and organizations, many of us profess to hold up the value of stewardship, of caring for the earth’s resources. But as some of us move more of our lives online, how much do really know about the real-world impact of our actions and data online lives?

When Google’s Gmail service launched, it advertised “never delete an email again”. Instead, you can archive the e-mail with a single click, and it will always be there in case you might like to find it later.

As part of the launch, Google was offering about 100 times more e-mail storage than their competitors. This was enough, they claimed, to never delete another e-mail in your life. This was a decisive moment that changed web-based e-mail forever. Competitors scrambled to dramatically increase their storage options so they could compete.

Something there bothered me. In the physical world, this is a way of thinking that no environmentalist would stand for— NEVER THROW ANYTHING ANYWAY AGAIN? The circle of life is broken, replaced with a one way trip from creation to permanent storage.

Are the rules for sustainability online really that different?

There’s been a belief that when we move activities online, we are being green. We laud “Going paperless”, and celebrate e-everything.

There of course some truth in the efficiencies of digital living. It’s certainly intuitive that’s less resource intensive to send an e-mail instead of a physical letter, or teleconference instead of flying somewhere for a meeting.

But along with some of these efficient uses of the internet, we’ve moved some of our unsustainable practices online without deeply questioning the impact of this.

While it may be efficient to send an e-mail instead of a letter, many of us now send and receive far more e-mails than we wrote letters. Our use of the internet has gone far beyond replacing physical tasks with efficient digital alternatives.

I’ll share what I know about the carbon footprint of our online lives now.

To talk about the carbon footprint of our online lives, let’s start with the the physical existence of the Internet. Websites and e-mail are served for computers all over the world. Many websites are now clustered in a relatively small number of large data centers.

Picture a data center as a dimly lit, windowless warehouse. On the concrete floor sits aisle after aisle of floor-to-ceiling stacks of computer, neatly set on identical racks, with a blinking lights on the front and neatly organized cable on the back. There is an incessant hum from thousands of spinning disk drives and fans to cool the systems. The temperature is comfortable, thanks to dedicated cooling systems for the computers. The aisles are even emptier of workers than it is at Lowe’s. A small number of people may be onsite to tend to the rare physical needs of the machines, but most people who use the systems could be anywhere. Like you or me, they could even be sitting at home in their underwear.

Already, the data centers that host major Internet sites are drawing more electrical power in the United States than our TV use. [1] Let me say that again: the electricity American’s consume to power to their Internet habit has surpassed the amount of electricity used to power to TV habit. And while we keep our TVs just a few hours a day, we expect websites to be available 24 hours a day, every day.

Data centers tend to be powered by traditional power sources, with a few exception who choose to use wind or solar to power their operations. Google has expressed sincere interest in greening their operations, but so far continues to focus on building out their infrastructure as fast as they can, with a plan to throw money at the sustainability problem, hoping for a solution later.

A scientist researched the energy consumed by a Google search and determined that executing just two Google searches would use enough energy to boil a kettle of water [2] Google refuted this claim, saying that this estimate was far too high. Google performed it’s own carbon footprint calculation of a Google search. According Google’s own estimate, it would take a 1,000 Google searches to equal the impact of driving an average automobile a kilometer, or 6/10ths of a mile [3]. Sending a search to Google isn’t just asking a question to a single computer. Clusters of super computers are used to calculate a response. The footprint of a search is small, but the number being executed every day is staggering. I’m sure Google was trying to present their environmental impact in the best possible light. It’s no wonder then that they didn’t cross reference these statistics with the number of searches that are currently performed each day. It’s estimated that about 300 million Google searches are performed each day.[7]

This means that according to Google’s own estimate, the daily impact of Google searches adds up to the equivalent of driving about 180,000 miles each day. Calculating this number was of my deciding points in preparing this message. It’s such a big number. Imagine if there were 180,000 less miles driven each day!

With some further research I was able to put this number in perspective. (I think it took less a thousand additional Google searches). The United States Postal Service logs an estimated 2.6 million miles each day, or about 15 times more. [6] Americans in total drive about 5 BILLION miles a day. [4] The impact of Google searches is statistically insignificant compared to this. To try to put this into perspective: If American’s were to drive one mile less per year, it would have more a thousand times more impact than if the entire world abstained from searching Google for a single day.

I don’t mean to diminish the original number: The daily impact of Google searches equating to 180,000 miles of driving in terms the carbon footprint. It’s still a big number and it would great to reduce it further. Comparing the impact of different activities we perform helps us to put things in perspective and prioritize what lifestyle changes could most effective. And we don’t always have to chose making one improvement at the expense of another.

The Google search statistic was an example of taking an action online. Life online involves more than just Google searches though.

Our online lives are also composed of data we generate or that is collected about us, sitting up there in the “cloud”, at these data centers. There are e-mail folders of archived messages. There are archived posts to mailing lists and forums, and photos of old summer vacations posted on photo sharing websites.

Our data has a cost to exist as well. Data that seems to be inactive is likely to be regularly accessed for maintenance like virus scans, causing an energy draw proportional to the amount of data involved. Any data stored online is likely backed up every day. Even inactive data is copied repeatedly to back-up tapes, causing additional power consumption.

What is this impact of this storage in context? I don’t know, but it’s clear that the more data is out there, the greater the cost to store it.

There’s so much data being stored about us, often not because we care about it, but because it benefits the corporations who are collecting it. The more data Google, Facebook and others collect, the more content they have for pages to serve ads on, and the more relevant ads they are able to display based on the data we give them.

So Google strongly encourages us to archive e-mail, not delete it, which would reclaim resources. Likewise, Facebook and many other sites have few or no limits on the amount of content you can post. Instead, they focus on infinite data structures, like Flickr’s “photostream”, Facebook “walls”, and the endless river of status updates on Twitter.

The design of these sites is not to encourage us to review all of someone’s content, or even someone’s best content. The design pattern we see over and over online now is to encourage an infinite streams of data, and have us focus only the most recent entries of the infinite streams, while meanwhile the old data is encouraged not to be removed and recycled, but to stay online forever for reference and profit.

It’s a hard problem to design tools that find the most relevant information regardless of whether it’s the newest or not. Google search tries to solve just that problem. The problem could be somewhat voluntary addressed if people took greater care to update the information that was posted online, or delete content we controlled that knew was obsolete.

As stewards of our online lives, we should apply the same kind of thinking we do about physical world sustainability to our online lives.

Re-consider allowing so much of your data to make a one way trip to permanent archiving. Cultivate your data like a garden. Something with finite boundries. Review the things you’ve planted online periodically. Throw away content that has rotted or expired over time. Prune out the typos. Trim and rewrite your best pieces so they can flourish.

Use your data gardening time to reflect on your past. You may ask yourself “Whatever planted the seed for that article in my head?” But you may also find some heirloom crops, still bright with flavor today.

Now let’s zoom out some. How can we profess to be good stewards of the earth, when we engage in activity where we don’t really know the impact?

Religious history has seen groups split over such questions. Should we use automobiles? Electricity? The Internet? The Amish stand out for choosing the simpler life, while other demoninations attempt to live “in the world but not of it.”

Communicating through the internet is just one example of lifestyle choices which create a more abstract existence, where the affects of simple daily activities touch back to data centers in California and factories in China.

To embrace this complexity while still prioritizing stewardship means taking on the responsibility of understanding the impact of our abstracted actions, from using the internet, to driving cars, to buying foreign-made products.

When it comes being a good steward of our online lives there are many ways to address the complexities and reduce our carbon footprint. Here are three specific practices that I use. The impact of each action may be small, but like a vote, the cumulative effect of small actions can add up to something big. The benefits of such practices go beyond simply reducing carbon footprints. Each one is a practice in mindfulness, that reminds us that our abstracted actions have real world impacts.

The first tip: I put our home cable modem and wireless router on a power strip. We turn the strip off at night and on in the morning. Not only does this save electricity, it also improves security by completely preventing outside access. It also reduces the amount of radio waves being broadcast through the house.
A second tip: When sending an e-mail that is primarily an attachment, I consider using the option to not save the message in your sent-mail folder. These messages are much larger than normal e-mails, and I already have a copy of the document on your hard drive, plus the recipients will also have a second copy in their Inbox, and likely a third that is saved to their own hard drive.
Finally, here’s a tip that could vastly reduce the number of Google searches, while at the same time finding what you are looking even faster. Top Google searches include queries for “YouTube” and “Facebook”. Instead of going directly to a site like “YouTube.com”, many people first type “YouTube” into Google and click on the first result. Using a bookmark for popular sites would save a small but repetitive amount of time and energy by going directly to the sites. A bookmark is not only efficient here, it makes that Google is not tracking your search and mediating your experience as pass through Google. You are saving yourself from seeing one more ad that day, which would otherwise be displayed in the right sidebar of Google as you click through.

Ultimately I think the wisdom of “less is more” that applies to being stewards of our online lives. You have the option to just not post something. Or Don’t sign up for some website. Or just unplug and go outside. Visit someone in person. Stewardship the old fashioned way has a beautiful simplicity to it.

How have you found satisfaction and success in being a steward of your online life? If you don’t use the Internet, or have even just avoided Facebook, what has it meant for you to chose this decision while so many others embrace it? What do you find at the intersection of our spiritual and digital lives?

References:

an EPA study stating that the data center industry devours 61 billion kWh of energy annually compared to ...about 275 million TVs currently in use in the U.S., consuming over 50 billion kWh of energy each year
Performing two Google searches from a desktop computer can generate about the same amount of carbon dioxide as boiling a kettle for a cup of tea.
the average car driven for one kilometer (0.6 miles for those in the U.S.) produces as many greenhouse gases as a thousand Google searches.
Americans drive 5 billion miles per day
The Dept. of Transportation estimates that Americans drive an average of 29 miles per day
The Postal Service operates a fleet of 219,000 vehicles, including 146,000 delivery vehicles…The average LLV is driven about 18 miles a day. (146,000*18 = ~ 2.6 million miles per day )
…299.83 million Google searches per day in May 2009
The book Planet Google was also a useful reference.

A vision for CGI.pm and CGI::Simple

2009-09-24T01:19:23Z

I’ve spent a lot time recently triaging bugs for CGI.pm. I’ve enjoyed the process, and respect CGI.pm as a widely used Perl module. I’m not in love all aspects of module. I don’t use or recommend the HTML generation features— I recommend using HTML template files and HTML::FillInForm for filling them.

Whenever I think about how I’d like to change CGI.pm,what I have mind is often the same choice that CGI::Simple made. There was a time years ago that I focused my attention on CGI::Simple and tried it in production, only to be bit by a compatibility issue, so I reverted back to CGI.pm. I don’t remember what the specific issue, and it’s likely been fixed by now. But the pragmatic point remained with me: CGI::Simple may have clean code and a good test suite, but it’s not necessarily free of defects and in particularly it lacks the vastly larger user base that CGI.pm has to provide real world beta testing.

]]> I recently took another look at CGI::Simple, it’s cookie handling implementation, and its bug queue. One thing became clear: CGI::Simple forked from CGI.pm in 2001, and they have not evolved in parallel since then. Each has had different bugs filed against it, with some issues fixed in one and not the other. They both have test suites, but they have evolved with different test coverage as new tests are written to respond to bugs filed against one particular module.

And, unfortunately for the better design of CGI::Simple, it is CGI.pm that continues to receive far more of the attention and updates. (Although to be fair, some of this relates to the HTML functions, which are intentionally omitted from CGI::Simple).

I would like to say that CGI::Simple is a clear path forward from CGI.pm if you are willing to let go of the HTML generation functions. Unfortunately, the current situation is ripe for running into subtle differences that have been created since the projects forked about eight years ago.

My vision for a solution is simple: CGI.pm and CGI::Simple should be maintained together. Where their features overlap, the combined project should have the best version of the documentation from both projects, the best code from both projects, and the combined test coverage of both projects. CGI::Simple is intentionally incompatible in a few ways in the name of better design, and I support that. Still, the projects should strive to maintain compatible whenever possible to make it easy for people to transition from CGI.pm to CGI::Simple. When a change comes in that could affect either module, it should be changed in both modules.

A great example of a possible collaboration point is the request to add PSGI support to CGI.pm. Ideally if the proposal is accepted, it could be added to both CGI.pm and CGI::Simple at the same time, with the same API, tests and documentation.

Almost 100 CGI.pm bugs closed, help with the 50 still open

2009-08-15T01:28:23Z

Get off the couch and pull your weight—
There’s CGI.pm bug with your name on it.

There were nearly 150 active entries in the CGI.pm bug tracker when I was approved recently as a new co-maintainer. As I had time in the evenings after the baby was sleep, I went through and reviewed every one of these bug reports. Many had already been addressed by Lincoln some time ago. Those were simply closed. Still, I found about 20 fairly ready-to-go patches, and those have now been processed and released today as CGI.pm 3.45. Whenever code changes were made, I also strived to make sure new automated tests were added that covered those cases. You may be surprised how many methods in CGI.pm have no automated tests for them at all.

Now there are still about 50 open issues in the CGI.pm bug tracker. For these, I have tried to use the subject line some summary indication of what is needed to move it forward, like “Needs Test: “, or “Needs Peer Review: ” or “Needs Confirmation”. Generally, I plan to wait patiently for volunteers to help with these. If you use CGI.pm, consider helping to move one of these forward.

]]> To make collaboration easier, CGI.pm is now on github. You are welcome to fork and send pull requests through there, although posting patches to the bug tracker continues to work file for small changes. The full CVS history has not been translated yet, but may be eventually.

CGI.pm has been in the Perl core since 5.4. With its maturity comes quirks. I’m not a fan of the HTML generation functions. I find the support for both OO and procedural styles awkward. As I have more time, I also hope to continue updating the CGI.pm documentation to promote more modern practices, and de-emphasize other parts of it, like the HTML generation functions and the procedural interface.

I would like to thank Lincoln Stein for building and releasing CGI.pm, and for maintaining it for over a decade. Many projects from CGI::Application to Movable Type depend on it. I also appreciate his willingness to allow for direct help on the project, and his receptiveness to the documentation overhaul idea. Lincoln is continuing his involvement. He completed the final review and release of the changes proposed for 3.45.

I understand that CGI.pm is broadly used, but like ExtUtils::MakeMaker, not always well loved. It’s true that some day it will be completely replaced by next-generation tools, and some reasonable candidates exist now. Until then, there are thousands of existing users will appreciate our collective maintenance of the module. Let’s get the bug tracker back down to zero!

Movable Type fork is an opportunity to harness CPAN

2009-06-23T16:21:51Z

Today Melody was announced as a fork of the perl-based Movable Type platform. I helped the Melody project as it prepared to launch, in part advising on how to best to relate to the Perl community. One of the stated interests of Melody is to refactor the project to use CGI::Application, which I maintain. Tim Appnel has already spelled out a vision of what a "CPANization" of Movable Type might look like, and I've looked in depth at what the initial steps towards using CGI::Application could be.

My own vision for Melody is a code base that's very focused on publishing and content management, with all the infrastructure outsourced to CPAN modules that are well-written, well-documented, and well-tested. The collaboration between Melody and CPAN would be a two-way code flow. While there are more CPAN modules that Melody could make use of, there are number of pieces of Melody which should be packaged as independent modules on their own and released to CPAN. One example is the great "dirification" that already exists in Movable Type. This is the functionality that turns any given string of words into a reasonable representation in URLs. It seems like an easy problem on the surface, but Movable Type has a sophisticated solution that takes into account what it means to do this well across many different languages. I also couldn't find any existing CPAN module which already takes on this problem space, so I started to extract this out of Movable Type myself and published a draft of String::Dirify. For that initial release, I ripped out all the fancy multi-language support, and there is still more significant work to be done to untangle this layer from from Movable Type. ( If you want to pick up that project and work on it, there's also some discussion of testing String::Dirify).

While Movable Type already had an open source release, I expect Melody to have a more adventerous evolution, and I look forward to it becoming a shining star in the Perl community, not just for the exterior functionality, but also because internals have an opportunity to become an example of best practices.

Towards a Pure Perl HTML::FillInForm: 61% tests passing

2009-04-28T02:50:44Z

I have a goal of distributing Titanium along with all of its dependencies. The vision is to have a pure perl stack, so it’s easy to unpack and start using the distribution, without worrying about binaries tied to a specific platform.

One stumbling block with this has been HTML::Parser, which requires XS and is dependency of HTML::FillInForm, which is used by CGI::Application::Plugin::ValidateRM.

I’ve been working with Ron Savage on a Pure Perl replacement for HTML::Parser, and I made good progress over the weekend based on his foundation. He created HTML::Parser::Simple, which is really is two major parts in a one. First, it provides a pure-perl HTML parser. Secondly, it also provides a specific use of the parser, which is store a representation of a HTML documentation as a Tree::Simple data structure.

In a future version, I’d like to see the Tree::Simple part split into it’s own module to clarify the parts, and to eliminate this dependency for people who don’t use it. For example, in the new sub-class I created, I don’t use the Tree::Simple object at all.

I made a short-term goal of getting all the tests for HTML::FillInForm to pass when we modify one line in HTML::FillInForm to say that it “isa” HTML::Parser::Simple::Compat” rather than being a sub-class of HTML::Parser. Then I run the HTML::FillInForm test suite and see how many tests pass and file. Currently our parser emulates enough of HTML::Parser to pass 61 percent of the tests.

The idea with “HTML::Parser::Simple::Compat” is provide an API that is compatible (enough) with the HTML::Parser 2.x API. I think the approach seems promising, but there is still more to do:

I have a focused on getting tests to pass, so the documentation is still poor
Perhaps my “Compat” sub-class should really be merged into the parent class. I’ll discuss this with Ron
There are few of our own tests, as I’ve been focusing on getting HTML::FillInForm tests to pass. Perhaps more of those tests could be “ported” into own test suite.
Performance and memory consumption have not been benchmarked.

If this project interests you, feel free to fork the project on github and start contributing code or other feedback.

fgdb: Free software to manage community hardware recycling

2009-04-03T23:50:44Z

Hardware recycling in Richmond took a leap forward last weekend. A small group of Richmond volunteers toured Free Geek Columbus. We learned much from visit. One valuable detail I’ll focus on today is that they run their organization on web-based, database-driven software system called “fgdb.rb”. The software is available on their intranet, allowing several volunteers to use and access the system at the same time.

fgdb.rb tracks hardware donations, volunteer time, recycling trips, and hardware distribution. fgdb.rb work with a neat tool called “printme” that takes an automatic snapshot of the all of computer system’s details, and uploads it to the database. This automates tedious data entry and creates a great reference.

I was pleased to find that “fgdb.rb” was available for free as open source software, and is designed to runon Linux. However, the documentation was lacking on details on how to get the system up and running on Ubuntu Hardy Linux, which is what we use on our server at our hardware co-op, and also on my laptop.

Free Geek Portland developed fgdb.rb for their own use and had tested installing the software primarily on Debian Lenny Linux. Now I have managed to install it on Ubuntu Hardy Linux, and have submitted a patch back to the authors to update the documentation to help others do this as well. You can see my current version of the installation instructions, but my changes should be merged into the main fgdb.rb git repository soon, and I recommend checking there for the current version.

I’ve also published a few photos from the Free Geek Columbus field trip.

Darcs vs. git annoyances: getting to know you

2009-04-02T02:09:23Z

I just saw a patch I created on github attributed to “mark@freekbox.(none)”. That’s nothing like an email I use and it’s annoying that git not only selected this as my identity to use in the commit log, but then didn’t even run it by me for a reality check. Now that I’ve committed that patch and pushed it to github, there’s no real way to alter that. Extra annoying.

By contrast, the first time you record a patch in a darcs repo, you are prompted for your e-mail address in a friendly way:

$ darcs record
Darcs needs to know what name (conventionally an email address)
to use as the patch author, e.g. 'Fred Bloggs '.
If you provide one now it will be stored in the file 
'_darcs/prefs/author' and used as a default in the future.  
To change your preferred author address, simply delete or edit
this file.

What is your email address?

Easy and pleasant.

Richmond Schools: Consider Thin Clients

2009-02-08T02:44:12Z

Richmond High School student Jonathan Ulrich helped to set up and test a thin client lab.

This is an open letter to the Richmond, Indiana Community School system. There is a school board meeting coming up to discuss how to fund technology upgrades with a dwindling budget. I strongly suggest the school system consider Linux thin client labs as part of the solution. Thin client labs are made with low-cost, low-power, low-maintenance stations and have many advantages.

A Linux thin client lab is already being used successfully in the area. Four years ago in Brookville, Indiana a thirty-seat thin client lab was set up at St. Michael’s School. Initial costs were kept low through low hardware requirements and the use of free, open source software. The lab is still in use four years later. Minimal maintenance has been required, including zero virus/spyware/malware infections due to the use of Linux.

Thin clients don’t need a hard drive, which are at the top of the list of the common parts to fail in a computer. Instead, every workstation pulls all the software it needs from a single server, meaning there is a one computer to maintain software on in the lab, not thirty. So St. Michael’s unplugged the hard drives in their machines, cutting down on noise in the lab, and well as reducing the energy consumed by the lab.

I recommend checking for yourself on this success story. For the administrator perspective, contact the Principal, Ken Saxon at (765) 647-4961. For the IT perspective, contact Mike Heins, who set up the system and maintains it: (765) 328 4479, (also at mikeh@perusion.net).

The use of Linux in Indiana schools is not new, either. In 2005 the state of Indiana launched a state-wide initiative to put Linux on the the desktop of 300,000 Indiana high school students. Locally, Northeastern High School has made significant use of Linux.

I’ve already hinted that thin clients have lower power requirements and can be lower maintenance. The hardware needed for thin client workstations is not special. In fact, old desktop hardware that would otherwise be discarded for being slow is ideal. In a thin client system, the performance is determined by the server, and the workstation needs just a minimal amount of resources to connect to it.

With these principles, I built a four-seat demonstration lab at my church, using three computers so old that a local computer store gave them to me. I paid only $50 for a memory upgrade for the server. As a thin client lab, these old computers came back to life and performed like modern desktops, although they ran Windows 98 in their former lives.

Because a school lab setting is ideal place to deploy a thin client network, there are several projects that focus on exactly this, and give away the required software. These include K12Linux and Edubuntu. Both are exceptionally easy to try out and install, from personal experience.

Pursuing thin client now is a strategic move that works towards the goal of the City’s Comprehensive Plan to be a “Sustainable City”. The plan is fiscally conservative and technologically advanced, with low impact on the environment and energy bills.

Mark Stosberg

Running Perl tests in parallel with "prove", with some exceptions

Plack::Request: not as read-only as it might seem

Reflections

Best practice for handling UTF-8 when percent-encoding?

Percent-encoding URIs in Perl

Percent-encoding with CGI.pm

Percent-encoding with URI::Escape

Summary of all known percent-encoding solutions for Perl

Recommmendations

generating HTTP headers: sorted or unsorted?

The cost of saving sent e-mail

Calculating the carbon footprint of e-mail storage

See Also

References:

Modifying PDFs so they open full screen

Stewardship and Sustainability of our online lives

References:

A vision for CGI.pm and CGI::Simple

Almost 100 CGI.pm bugs closed, help with the 50 still open

Movable Type fork is an opportunity to harness CPAN

Towards a Pure Perl HTML::FillInForm: 61% tests passing

fgdb: Free software to manage community hardware recycling

Darcs vs. git annoyances: getting to know you

Richmond Schools: Consider Thin Clients