Open Source Enlightenment

(My thanks to Audrey Tang for this lyrical transcript of my talk at OSDC.tw, to Macpaul Lin for the video, and to Chia-liang Kao for proofreading the Chinese translations in my slides.)

Over the years, I’ve started thinking that participating in the open source community is like traveling on a path, toward becoming not only better programmers, but also becoming better people by working together.

You might think of it as a path toward enlightenment, growing ourselves as human beings. So what follows is really my personal philosophy which I’d like to share with you.

The first thing is this: The most important part of every open source project is the people. While code is important, the center is always the people.

There are different kinds of people involved in a project: People who code, who write documentation, who write tests. People who use your software, too, are just as important for a project.

Also there are people who work on the software that your project uses — you’re likely using projects from other people in the upstream, and you might want to send them a patch from time to time.

Or maybe you’re writing a library or a module, and so other people will be using your software, and communicating with you as their upstream as well.

So why do people work on open source software? This is a very important question to ask, in order to understand how open source works.

For people’s day jobs, they may be working with software already. And why would they take the extra effort to work on open source? Part of it is that it involves working on exciting things and new technologies.

Sharing is also a large part of it; as we share with each other, we increase the amount of fun for everyone working together on an open source project.

People also work on open source in a spirit of giving to others; in doing that we’re reaching out as human beings, and this is a very important part of being human.

There are many rewards, too. A big one is respect: As we create something new, draw people in, and share software with them that they can work on too, they recognize who you are and what you are capable of, which gives you a sense of accomplishment.

Conversely, it means that we want to make sure that we show respect to people joining our projects in any way we can, because it helps them to stay involved.

Another important aspect is appreciation; as people publish their work, if you talk with them — Even just a simple thank-you email message saying “this meant a lot of me”, it helps bring about a culture that keeps everybody motivated.

Credit is also important. As you are presenting a project, be sure to mention other people around you, saying “this person did such a wonderful thing”, so we can build a feeling of community together.

One of the things that keeps people interested in open source is that, as we work together, we become stronger and can do more.

Part of it is simple math: 2x people makes at least 2x code, and 3x people makes 3x code, although there is much more to it than that.

When we work together, we can make each other stronger and better — part of that is encouraging each other; as you see people working on a very difficult problem, you can encourage them saying “you are doing great, and I see you will do great in the future”.

You can empower people just by talking and sharing with them.

And then also there’s the fact that, when you have many people together, they’ll have different sets of skills. When you are working together, maybe you know the five things the project needs, and they know the other five things, and so you have the complete set of skills to finish the project, which wouldn’t be possible if either of you worked alone.

So the effect is not only a linear increase in productivity; there’s a multiplication effect when people start working together.

Encouraging each other to look beyond, to look into the future, is also important — We can all inspire others to solve interesting problems. Sometimes just saying “I have an idea” is enough for someone else to make it into reality.

Sometimes you’d look at what someone else is doing — you have not done all the work, but you have the critical idea they needed, and so with that idea they can reach out and go much further.

The key thing about working on open source is that we’re not just standing alone. When you are working with other people, the main thing you’d want to improve is your communication skills.

We communicate about the plans we have: How we want to make the software, personal plans such as a feature you want to work on, and so on.

One of the things I observed in open source communities is this: People often have good plans to create software, but they sometimes clash and fail to communicate with each other about plans. If you work on one plan alone, without communication, you may end up hurting people working on other plans.

So it’s like a hive of bees — a constant buzz keeps us all functioning.

We’ll also often communicate about possible futures: What’s the best way to solve a technical problem? When this happens, you may communicate in a way that’s contentious and angry, making it very hard to make actual progress.

One of the things we’re learning in our process is how to embrace all possibilities. Keep working on the possibility you’ve imagined, but remain fully open to other possibilities other people may have.

And as you make progress, you’ll also be communicating constantly about what you have done — There’s email, there’s twitter… there are many ways to let people know about your progress.

Sometimes we may feel shy, or not wanting to be seen as bragging. But that’s not what it is! It’s good for the project, and for the people as well, because they can learn from what you have done.

Another aspect of communication skills is the ability to ask questions. The advantage of having a community is that some people might have solved your problem before, and asking a question on a forum or IRC may save you days of work.

In the same way, when others are learning, you can be responsive to them too, instead of putting them down like answering with “RTFM” for simple questions.

It’s true that answering “RTFM” maybe save you a bit of time, but it is also teaching that person that they shouldn’t ask those questions in the first place. That is not what you want to teach people at all — you want to teach them to communicate with others.

Also, learn how to make answers that are helpful to people, and help them see that they can also walk down the path as well, and take the path further in the future.

Sometimes you do have to criticize people; we should be open to many ways of doing things, but sometimes one technical solution really is more correct than others. However, the best way to get people to change their ways is to answer them kindly, so they can be open to learning from you.

You have to show some grace, even to people who do not respond very well. Some people may be harsh with you, but this is also part of the path. Sometimes it helps to have a thicker skin, and even in situations when other people should have said things nicer and better, maybe there’s a bit of truth in what they are saying, and you can still learn from that.

From this perspective, even if they speak in a way that is not polite, you can still respond politely.

The other half of communication is not talking, but listening. Instead of telling others what we think, sometimes all that’s needed is just sitting very quietly, and let others talk.

It’s not just listening, though — it’s important to have empathy. As the saying goes, “If you really want to understand someone, you have to walk a mile in their shoes” — perhaps so you can get the blisters they have experienced.

Now, some people think you have to be a genius to work on open source software, but that is simply not true. There are people like Larry and Guido and Linus, yes, but there are also so many different kinds of talents that any projects needs, too.

And no matter how smart you are, it’s important to stay humble. Because with humility, you will be open to other people, and see new ways of doing things. Humility lets you welcome other people into your project. Pride, on the other hand, is essentially telling people “I don’t need you; I can do things my way.”

By being humble, we also welcome people with diversity of genders, of different cultures, creating a richness in open source by opening to different kinds of people.

The diversity also appears between different projects; it’s almost like languages and cultures of different countries. For example, the community around Linux, Perl, Ruby, and Python all communicate and collaborate differently.

And by being humble with each other, maybe we can see that our project is not the only way, and maybe we can appreciate the ways of other communities.

Now, open source is not all about fun — it’s fun, of course, but it’s also a responsibility. When you agree to participate in a project, you’re taking a weight on your shoulders, and it’s a good thing, as it teaches us to improve ourselves and become better humans.

But life can get in the way — significant others, parents, children, jobs — we may accept responsibility for a time, but there may also be a day where we can’t carry so much responsibility anymore.

So there is a cycle, where you start by assuming more and more of a role in a community, and as life goes on, you gradually take on less and less responsibility. This is entirely natural, and it’s bound to happen in a project’s life cycle.

So it’s worth keeping this question in your mind: “Who will continue my work when I no longer have the time?”

To make sure other people can continue our work, we can think of it as a continuous process: Teaching and sharing the knowledge we’ve learned, and at the same time learning more and more from other people — a continuous process of gaining and sharing knowledge.

Finally, as you work on open source, please be happy, with a smile on your face, and make other people happy! Because this happiness is what gives us the power to make great things.

Do you feel happier now? 🙂

Tody Task Manager

Failing to find any free software task manager I could live with, I created my own over the December holidays. I called it “Tody”. It’s a simple GUI app, focused on quick searching, editing, and tagging for tasklists. The file format it uses is identical to the plain text format used by Gina Trapani’s Todo.txt command-line tool and Android app, it even loads preferences from the Todo.txt config file. Since the file format is plain text, tasklists can be shared between machines (or users) over Ubuntu One or Dropbox.

I created it using Rick Spencer’s Quickly templates (GTK, Glade, and Python). I went for a streamlined workflow for the way I use tasklists, so I’m curious if it will map well to others. It appears as a simple text file, with a search box at the top of the window. Clicking on a tag performs a search for the tag (these are similar to Twitter tags, any word that starts with “@” or “+”). The list sorts tasks by priority (marked with “A”, “B”, “C”, etc) and then alphabetically. When the list is limited to search results, the search terms are highlighted in the tasks.

Clicking on the text of a task brings up an editor window, with a checkbox for “Done” tasks, a field to edit the task, and clickable palettes for task priorities and all the tags you’ve used previously in your tasklist. It’s streamlined with shortcuts, so typing Space, Enter marks a task as done, saves it, and closes the editor window.

I’ve started using Tody as my primary task manager, after dumping all my old tasks from other task managers into one text file. I’d like to tweak the search feature, right now it does a completely literal string search, but I’ll change it to split up search terms (so it’s not sensitive to order of terms). Then the next step is to link it up with my Todo Lens, so the edit window for Tody pops up as the action for clicking on a task in the Lens.

The Tody app is up on my PPA, let me know if you try it out and have any requests for features that fit your workflow:

https://launchpad.net/~allison/+archive/ppa

Free Software for Task Management

I am perpetually trying out online task management tools. My never-ending quest is to tame the massive sea of things I should be doing at any given moment, both making sure that important tasks don’t get lost in the mix, and to extract a reduction more closely approximating “the most important thing to accomplish right now”. My two favorites at the moment are Thymer and Rypple, but neither is perfect.

I like Thymer’s simple task creation, twitter-like tagging of tasks, and the smooth drag-and-drop motion for prioritization. But, at the end of the day, it’s just a massive web page of “things I should be doing” and gives me no assistance in taming the beast. I have to manually prioritize each task, and if I want the priorities assigned to tasks to be at all relevant, I have to go on manually gardening them every day. And, while task creation is as easy as tweeting, task editing is a clunky collection of buttons and drop-down menus. The tags are handy in small numbers (and projects really are just tags with a slightly different display), but any more than about 10 unique tags/projects across my whole data set becomes a jumble at the top of the screen and not at all helpful in finding anything. Thymer offers some reporting features, but I never found them particularly useful.

I like Rypple’s social features, it’s got a good take on sharing thanks and feedback, and the 1:1 pages (a collection of tasks you share with another person) are incredibly useful for weekly meetings with co-workers. I like the organization of tasks by goal rather than by project, it encourages grouping tasks into larger sequences toward an overall purpose. But, I found that I still needed some goals that were really just projects or a collection of semi-related tasks, so the construct was a little artificial. Rypple offers a tagging feature, but tag links don’t do anything useful (like take you to a page listing tasks with the same tag), and a task can’t live in more than one goal at the same time, so there isn’t really any good way to pull up a group of cross-cutting tasks. And, Rypple also gives me little help in managing the mass, though it has drag-and-drop priority setting similar to Thymer.

The worst thing about both of them is that they’re neither open source nor open data. Philosophical considerations aside, this is an immediate practical problem, since my access to Rypple was only a free trial which is now ending.  I started with the best intentions of only putting in a few things to try it out, but it quickly became an integrated part of my working life, and I now have well over a hundred little individual blobs of data (tasks) that I’m tracking there. Because it’s not open source, I can’t fire up my own instance of it. And because it’s not open data, I can’t get a dump of my tasks. So, I’ll have to manually copy every bit to some other task management system. Which means I’m in the market for a new task management tool, with a very immediate enlightened self-interest in picking something that’s both open source and open data.

Yesterday, I tried out Todo.txt. The biggest appeal is the simple open data format, so simple that it would work just fine as a manually edited plain text file. But, it offers a GPL licensed command-line client for easier task creation, searching, sorting, grouping by project, priority, or “context” (a notion from “Getting Things Done“). It also offers a GPL licensed Android client, which is in the process of being ported to the iPhone. On the downside, it doesn’t offer any collaborative features, so I can manage my own tasks, but can’t share tasks with others, or even provide visibility to others on a subset of my tasks or projects. And while creating tasks on the command-line is clean and simple, actually viewing/managing my 100+ tasks on the command-line (or Android client) feels a bit like viewing an elephant through a pinhole. It doesn’t have a desktop GUI client, though the wiki offers some suggestions on ways to integrate the simple plain text format into other desktop tools like Conky. The results weren’t thrilling (not really any better than the command-line), but they did give me an idea: how about a Unity Todo Lens?

I spent a few hours hacking on that, parsing the Todo.txt format in Vala and displaying the results in a Unity Lens with a general search box and filters for Project, Priority, and Context. I’m pleased with the result for a short experiment, but there are some drawbacks. The Lens really wanted my filters to be statically compiled in advance, while I wanted to create the filter sets on-the-fly from the Todo.txt file (i.e. let me filter by Projects that are in my tasks, not for some list of projects determined in advance). I may be able to hack around that with more time or a Python Lens instead of Vala. Also, a Unity Lens is a great interface for searching tasks, but not great for managing tasks. There’s only one “action hook” for a task, when you click on the icon/title. You can make that one action do anything you want, but it’s still only one action. I could make that one action mark a task as done (that seems most logical), but I’d still have to go back to the command-line to add new tasks, and edit task descriptions, priorities, projects, contexts, etc… Which takes me back to the original problem that the command-line isn’t a great interface for those tasks. What I really want is a slick, simple GUI client that the Lens could launch whenever a task is clicked in the search interface. Possibly a project for another weekend.

That’s all the time I have to work on the idea right now. While I leave it sitting for a bit, any suggestions on free software+open data task management tools you love? Or hate?

Appreciation for Kees Cook

Today is Ubuntu Community Appreciation Day, a new tradition in the Ubuntu community started by Ahmed Shams El-Deen of the Ubuntu Egypt LoCo. I’d like to take this opportunity to show appreciation for Kees Cook, who many years ago took time out of a busy conference to teach me how to build my first .deb package. That welcoming spirit — that patient recognition that every green newbie has the potential to become a future valuable contributor — is a key part of community strength and growth. It’s a pattern I emulate, and a gift I repay, by welcoming and mentoring other new developers. Over the years, Kees has demonstrated sane, sensible, calm, and wise technical leadership at OSDL (now known as The Linux Foundation), on the Ubuntu security team, and more recently on the Ubuntu Technical Board. There are many reasons I have confidence in the future of Ubuntu, and he is one of them. Thanks, Kees!

I’d like to thank the entire Ubuntu community for renewing my faith in the humanity of free software. When I stumbled on Ubuntu all those years ago, I had already been working in free software for what felt like a century, and was…well, tired. Your joy and delight in bringing free software to the world inspired me, and restored my passion for contributing. The heart and soul of free software is people like you, changing the world for the better every day. Thank you all!

Mythbusters – UEFI and Linux (Part 2)

Following up on my earlier post on UEFI and Linux, I got access to an identical system to the one with the original problem (an HP S5-1110) this week to do some install testing with various scenarios:

1) When I run through the standard install process with the Kubuntu 11.10 amd64 CD, I get exactly the same problem as James: I end up with a machine that has Kubuntu installed on a partition, but will still only boot into Windows. (I also get an explicit error message during the install saying “The ‘grub-efi’ package failed to install into /target/. Without the GRUB boot loader, the installed system will not boot.”)

2) Installing from the Kubuntu CD and wiping the HD has the same problem as (1), and the same error message.

3) Installing from the Ubuntu 11.10 amd64 CD into the same dual-boot configuration as (1) also won’t boot the Ubuntu partition, but it gives no explicit error message about the grub install failure.

4) When I install from the Ubuntu 11.10 amd64 CD and completely wipe the HD and replace it with Ubuntu, the install works perfectly, and the machine boots into Ubuntu afterwards with no problems. I can also install the ‘kubuntu-desktop’ package on the working system, and get a working Kubuntu desktop. This tells me that we’re not dealing with a UEFI or hardware compatibility issue here, just an issue with partitioning and the bootloader. Which is what James and I suspected last week, but it’s nice to have explicit confirmation (without wiping his friend’s machine).

5) Back to the Windows/Ubuntu dual-boot scenario in 3. Installing EasyBCD doesn’t quite work. It does give me a prompt in the “Windows Boot Manager” to choose between Windows and Ubuntu, but when I choose Ubuntu it just takes me to the grub prompt. That’s progress anyway. At the grub prompt, I type:

grub> root (hd0, 4)
grub> kernel /boot/vmlinuz-3.0.0-12-generic root=/dev/sda5
grub> initrd /boot/initrd.img-3.0.0-12-generic
grub> boot

And, it boots fine from the Ubuntu partition.

That’s all the time I had so far. A few observations about the system as it shipped from the factory. Windows is booting using a custom bootloader, the Windows Boot Manager which bypasses UEFI. In the dual-boot configuration that doesn’t work, the UEFI “BIOS” configuration and the efibootmgr command-line utility both recognize that the machine has a UEFI boot option for “ubuntu”, but choosing that during startup from the boot options still diverts straight to Windows. The machine didn’t ship with GPT partitions (which are one of the advantages of UEFI), instead it shipped with an old-fashioned MBR partition scheme (limited to 4 physical partitions). The working Ubuntu configuration (total machine wipe) does set up proper GPT partitions.

Quixperiment: Ubuntu and iPod

I have an old iPod that I occasionally use on car trips, but haven’t really modified in years (it mostly sits on a shelf). This morning I decided to play around a bit with hooking it up with my main Ubuntu desktop. I found a good list of options for managing an iPod in Linux on Wikipedia, and decided to try out both gtkpod and Rythymbox. Both seemed to work pretty well for interfacing to the iPod, no a super-shiny interface, but usable. A slight advantage to gtkpod, because it displayed my Smart Playlists, while Rhythmbox only displayed the static ones. Between the two, I can imagine using Rhythmbox as my primary music player, but would probably only use gtkpod for directly managing the iPod.

I copied my iPod music library over to Rhythmbox’s local library, just to try it out. It copied 3,249 tracks out of the 3,359 that were on my iPod. I got a few errors about duplicate files during the copy, all with generic file names like “01 – Track 01.mp3”. There were ~4-5 CDs like this, each with ~19-25 tracks, so that seems to account for the missing 110 tracks, though I didn’t keep exact notes, or do an exact comparison to see which files were missed. I’m guessing a handful of CDs I had loaded on the iPod were ripped with generic file names rather than specific titles, and that the iPod was separating them by directory structure, while Rhythmbox was loading them all in one directory so the file names conflicted. Just a guess, I’ll look into it more later if it ends up being useful.

Things I wish for in Rhythmbox:

  • The ability to copy a playlist from the iPod to the local music library, instead of recreating it.
  • The ability to synchronize my music and playlists between different computers/devices (will look into Ubuntu One for this later, it has some relevant features, though possibly not yet the full user journey I’m looking for).
  • A way to split up my local library into Music, Audiobooks, and Language Learning. Shuffle mode is pretty useless when it brings up random chapters of “The Hitchhiker’s Guide to the Galaxy” or snippets of Afrikaans language drills. I found suggestions that it’s possible to configure multiple Libraries for Rhythmbox in gconf even though it’s not displayed in the GUI, but there was no ‘library_locations’ key in /apps/rhythmbox, so I’ll have to poke around a bit more later to see if it’s still a valid key in current versions of Rhythmbox. (Separating libraries is a problem on the iPod itself, so this is just a same-old existing irritation repeated in a new piece of software.)
  • A shinier user interface, that makes it easier to find artists, albums, or songs I want to listen to.
  • More informative error messages when failing to copy files.
  • I found one work-in-progress on integration between Rhythmbox and the Music Lens, I’d like to see that complete.

Mythbusters – UEFI and Linux

A recent blog post about a user who was having trouble installing Ubuntu on an HP machine, sparked off an urban legend that UEFI secure boot is blocking installs of Linux. To calm FUD with facts: the secure boot feature hasn’t been implemented and shipped yet on any hardware. It was introduced in the 2.3.1 version of the UEFI specification, which was released in April 2011. Hardware with secure boot will start shipping next year.

It’s important to distinguish between UEFI in general and the new secure boot feature. UEFI has been around for a while, starting its life as the “Intel Boot Initiative” in the late ’90s. It has a number of advantages over old BIOS, including substantially faster boot times, the ability to boot from a drive larger than 2.2 TB, and the ability to handle more than 4 partitions on a drive. The UEFI specification is developed by the members of the UEFI Forum, a non-profit trade organization with various levels of free and paid memberships. UEFI is not a problem for Linux. At the UEFI Plugfest in Taipei last week, Alex Hung (Canonical) tested Ubuntu 11.10 on a number of machines, with success even on pre-release chipsets. The few failures seemed to be related to displays, and not particularly to UEFI.

The secure boot feature of UEFI is a concern for Linux, but not because of the specification. The features outlined in the 2.3.1 specification are general enough to easily accommodate the needs of Linux. But, within the range of possible implementations from that specification, some alternatives could cause problems for Linux. For full details, I recommend reading the two whitepapers released by Canonical and Red Hat and by The Linux Foundation. The short version is that secure boot uses signed code in the boot path in much the same way you might use a GPG signed email message: to verify that it came from someone you know and trust. The beneficial ways of implementing this feature allow the individual (or administrator) who owns the machine to add new keys to their list, so they get to choose who to trust and who not to trust. The harmful ways of implementing this feature don’t allow the user to change the keys, or disable the secure boot feature, which means they can’t boot anything that isn’t explicitly approved by the hardware manufacturer (or OS vendor). This would mean users couldn’t just download and install any old image of Debian, Fedora, Red Hat, SuSE, Ubuntu, etc. So, there’s real potential for a future problem here, but we’re not there yet. At this point, it’s a matter of encouraging the hardware companies to choose the beneficial path.

I’ve been chatting with the Ubuntu user who had the install problem, to see if we can find the real bug. It’s a friend’s machine rather than his own, so he doesn’t have easy access to it. I’ve arranged to get access to a similar machine next week to play with it. I’ll post back here if I find anything useful or interesting.