Welcome to my corner of the web. Here you'll find my ramblings about faith, church, drupal, Geeks and God (my podcast), and my other unrelated interests.

While you can subscribe to all posts here from the Subscribe link on the right, there are two other main feeds. There is the drupal and other technology feed along with the faith and church feed.

Scaling The Core Development Process

Posted on: Tue, 2009-09-15 08:46 | By: matt | In:

Screaming ImageFor many years the Drupal core development process has served us well. With Dries and a co-maintainer committing the changes of the community we have crafted a product we love and hate to use and a community we are involved with. But, the success of Drupal is stressing this 2-tiered development system. After talking to numerous leaders in the community here are a few suggestions as to how we can scale this system and relieve some of the stress for Drupal 8.

The Strain

The Drupal community has grown in size and the Drupal core package has grown in size and complexity. For anyone now sure what I mean just download Drupal 7 and compare it to Drupal 5 or 6. In many ways I think all this new stuff in Drupal 7 is good. But, we now have two big stresses.

First, the community has grown in size. Drupal core has hundreds and hundreds of contributors. Angie has done an incredible job being accessible to people who need to talk about core development. But, hundreds of developers is a lot of cats to herd. Even with Angie being online 24 hours in a day (is she really a robot) there are core developers who feel like they need more support from a core committer. In some cases their concern is very valid. There are a lot of areas to support, which bring me to my second strain.

Drupal has gotten a lot more complex. The subsystems in Drupal core (like the database layer, theme layer, etc.) are in many cases the same size and complexity of other open source projects. The core committers need to spend time working in all these subsystems and with so many of them that is a lot to cover (especially with day jobs and loved ones who need time). While they have worked really hard to do this and have tried their best there isn't enough core committer to go around.

We have ended up with a lot of passionate, frustrated, hope filled developers.

Easing The Strain

To ease the strain here are 3 suggestions. In good open source spirit I hope others will build upon them and create something better.

1. More Core Committers

For the sake of my examples here I'm going to use the front end developer community. It's a nice functional block.

Add core committers for different functional areas. So, add a core committer for front end development. This person would do core committer work for the theme system, JavaScript and style handling, and other front end related aspects of Drupal core. The duties of this person wouldn't be to just work on these systems but to keep a birds on view on Drupal core to know how changes in one system are interfacing with others.

The person responsible for this functional area should, also, be an expert on that functional area and someone who isn't trying to implement their system and way of doing things. I am not looking for the role of core committer to really change.

2. Top Level Overviews

Drupal is known for not having a road map. In many ways this is great. It means that anyone new coming in with a great idea can chase it. We need to keep that open. But, road maps do have one nice thing. They provide a top level overview of what's happening. There is no nice place to find that in Drupal and trying to figure that out for yourself in the issue queues is daunting.

I'd like to see a top level overview of what's happening in Drupal. Ideally those in the Maintainers.txt file alongside the core committers would keep and eye on what's going on and update the overviews. We could even start this now by breathing new life into the Community Initiatives.

Making these top level pictures of what's going on more easily available will make it easier for people to find what's going on.

3. Create Pictures of Where We Are Going

Some recent initiatives, like d7ux and the new database layer, have shown that if we look at more far reaching changes and what they might look like we can come up with some great ideas. Lets do more of them. To do this well we need a place to virtually do this and share what people are doing with others.

For example, we could list the "ideas" for the Drupal theme system on the theme system overview page. These could link back to an idea incubator where they are hashed out in a constructive information gathering manner.

Time For A Change

These stresses are a good problem to have. They are the result of success and growth. Just like so many problems we've come up against in the past we can tackle these.

Comments

#1 old story

1. same question, same answer: no. the real problem is that we need more reviewers, good reviewers. It is not the core committer responsibility to review the patches, so it won't ease your frustration. But, you can be the part of the "reviewer team" for the front end subsystem, you can decide which patch is good or not, and you do not have to apply for this role, just read and answer the issues.. Are you going to help out?

2,3. who knows
think about the changes in D7 which were planned, but never got implemented.
now, think about the changes in D7 which were not planned, but got into D7.
there are many many ideas, everyone can write his own list, and anyone can influence the path of Drupal.. If you do not like where it is going, then you need to realize it could be probably your fault, as you did not spent enough time to help..

#2 More than RTBC

Regarding #1, A core committer needs to do more than commit and review issues that are RTBC. Yes, we need more reviewers. Core committers have done a great job with the RTBC queue. But, core committers need to give some direction on issues that are not RTBC.

For example, what if there are 2 possible approaches to an issue. The team takes it all the way to RTBC but the core committer would prefer they had taken the different route. Now, the core committer comes in at the end and tells them to go down that second path. You now have a lot of frustration.

Core committers need to provide direction on issues as well.

DBTNG would never have happened without core committer direction. There was a decision on how to do the overlays in D7 and one path was picked over another. These kinds of decisions need to be made by core committers and do on a regular basis. But, there are a lot of issues that die waiting for core committer feedback. This is the part of the process where we need more of them.

Also, if you look back on previous versions of drupal we had more than 2 core committers. While drupal has scaled up we now have less available core committers.

Regarding #2 and 3, these are just different ways to view what's happening in Drupal. Visualizations if you will. #2 is a different way to look at what's happening. #3 is an encouragement and basic idea at looking at changes in the future. Many of these would not go it. It's idea sharing and incubation.

For these It's a matter of working in ways other than the issue queues for ideas and overviews because issue queues work for only certain groups of people. Others they turn away and we need to embrace some different work flows.

#3 Amen!

If you want a perfect example of why this sort of process change is needed, look no further than: http://drupal.org/node/432440

Schema API is right now lagging far behind the rest of the DB layer. It's API is old-school, it has limited capabilities, it's nearly useless outside of update hooks, and it's totally out of sync with the rest of the system. In April, I posted an issue laying out two different ways we could take Schema API. I asked "OK, so which do we want to do?" because as Matt says above no one wants to spend dozens of hours on one approach only to be told later "nah, let's do the other one instead". You'd charge a client extra to tell you that, but I can't bill Dries for my core development time. :-)

That was in mid-April.

I personally asked Dries for feedback by email.

I personally asked webchick for feedback in IRC.

I emailed the dev list pointing people to the issue.

I got involved in a lengthy discussion on the dev list, asking "so as DB Maintainer, what the heck do I actually DO? Do I have the authority to just make this decision and not have it overruled on a whim later?"

We invented a new tag, "Needs Committer Feedback", to flag issues that were stalled on getting direction from someone higher-up.

In mid-August, I finally gave up on ever getting any sort of feedback or acknowledgment whatsoever from the core committers and started working on one of the approaches, with a stance of "if they don't like my decision they can suck it, they had their chance". (That is, of course, a horrible, horrible attitude to have when writing code or anything else.) And once I finally got into the code, I realized that it was likely to be a much larger task than I had anticipated. And with only two weeks left before code freeze, two weeks in which I still had to prepare DrupalCon presentations and pack and so forth, I knew I didn't have the 100 hours to give it in that short amount of time so just gave up on it.

If I'd gotten any sort of direction at all, or a statement that I could set that direction, any time in that four month period, then we could have a vastly improved schema API and, potentially, much of schema.module in core. Instead, we got nothing.

That's not because our core committers deliberately ignored it, but because they both have jobs, families, and about 1000 other issues vying for their attention. webchick in particular had her attention focused on on the D7UX work, which is hardly a bad thing. But really... there's no reason whatsoever why the pinhole through which CSS and layout changes need to flow has to be the same pinhole through which the guts of the database layer need to flow. They're about as independent of each other as you can get. Why does the same person have to be the roadblock for both? They can absolutely happen in parallel without impacting each other.

The bottleneck here isn't code review, it's architectural review. And that's not something that can be solved by just "throwing people at the problem". It has to be the right people, people who are empowered to actually make decisions and resolve bikeshed roadblocks.

Matt's suggestion of adding more core committers for different broad areas, so that webchick could focus on UX stuff and ignore my database ramblings while someone else could work with me on the database layer and ignore bikeshedding over the toolbar, is one possible solution.

Another possible solution (and these are not mutually exclusive) is to allow subsystem maintainers to actually maintain; that is, tell me whether or not as DB maintainer I even am able to make that sort of decision and make it stick. Even a year after I became DB Maintainer I still have no idea what the heck that means. If maintainers had actual authority to make decisions rather than praying that somehow the bikeshedding and overworked committers would magically come to a conclusion, we would have been able to proceed on the schema updates five months ago and we wouldn't be stuck with a disjoint API for Drupal 7.

One way or another, the major problem is that too much responsibility is stuffed into too few people. That responsibility, and the matching authority that goes with it (because those two have to come together), needs to get spread out more or else Drupal's forward motion is limited to the available spare time that 2 people have. That's not sustainable.

#4 roles

everyone has the authority to decide.

if the core committer don't like the decision, then I (and other reviewers) can tell why it is the good solution. Even a core committer can be easily convinced. Do not expect that someone almighty will tell you what direction you should take with the db layer. It is your idea, your decision, you just have to defend it (against reviewers) if required.. Look at the core committer as one more reviewer, and all your concerns are gone..
We can set up a bot with a rule: If an issue stays in RTBC for 3 weeks, then commit the patch. The core committer shortens the 3 weeks, and we have an extra reviewer who really cares about the release.

you can delete the "Needs Committer Feedback" tag, it won't help you solving the issue you have. Your issue "Needs good reviewers".

#5 Read What Larry Wrote

Larry Garfield provides a great answer to why a core committer has a larger responsibility in his comment below. Many times before you have code reviewed you need an architecture reviewed and blessed by the core committer.

Realize, there used to be 3 core committers for a release (drupal 5 and 6). Now there are 2. Asking for another one isn't a crazy or new idea. Asking for certain core committers to have areas of speciality is the new part. Personally, I would like a front end dev to be concerned with UX, the theme layer, JavaScript, and some other stuff that could use this type of touch.

I don't debate that we need more people giving good reviews. This is something that is sorely lacking. But, it's a separate issue.

Having more core committers and them having functional areas is something that doesn't disrupt the flow of core development, it doesn't really change the process, and it's one that has support of a lot of thought and people leaders in the community. Including people with their names in the Maintainers.txt file.

#6 still no

I never heard of 3 core committers for a single release. Are you sure?

No, it is not a crazy idea for asking for more committers, I've did it also. http://drupal.org/node/217702#comment-717983
But I've changed my mind, because the core committer really should do less reviewing, it is just too much. This should be achieved only with good reviewers. See Gábor's comment, he clearly answers the problem. You need to realize that you can be the one who makes the final decision in specific areas without being the core committer.

people with their names in the Maintainers.txt file are the decision makers, but you can't demand from anybody, nor maintainer nor core committer that they will lead your issues..

Create a team of reviewers for front end issues, and you will be much more satisfied with the outcome than with an extra core committer.

#7 Drupal 5 example

For example take Drupal 5. We had the core committer of Dries, Steven, and Neil Drumm.

I'm not asking for anyone to lead my issues. And, if you ask people in the Maintainers.txt file what they can do you'll get some blank stares. See Larry Garfield's comment on this thread. He's had trouble with decision making on the database layer of which he is the maintainer.

Before you keep defending this point try checking with some of the maintainers and other community leaders. Take a look at the landscape a bit. Please.

A team of reviewers isn't going to help with some things. Sometimes you need direction from someone with authority (core committers are these people) before you go down a long path of work to make it worth your wild. Reviewers hold an important role. But, they can't do everything.

Additionally, for the area of front end development we could really use someone with expertise committing and guiding those issues. Steven used to fill that out quite nicely. Angie has many very smart people around her to help with it. But, we could really use an actual core committer who understand front end development.

#8 Untrue

people with their names in the Maintainers.txt file are the decision makers

Oh that such were true...

As I said above, I've been asking for a year to be told whether or not I had decision making power on anything. I spent a week on the dev list asking a very simple question: So do I get to make judgment calls on the database or not?

There were only two answers that were ever given. Steven Peck referred to an ancient book page that says "Though not directly making decisions, maintainers have informal responsibility for a designated portion of the core". What the heck does "informal responsibility" mean? It means nothing. But it does say that no, maintainers can't make decisions. Which means that maintainers are essentially useless.

webchick finally responded that being a subsystem maintainer means that we get to jump ahead in line when we need to talk to her in IRC about something. That's great, and it's at least an answer which is more than we ever got from Dries, but it still doesn't mean there's any sort of authority or decision making ability there.

So no, Maintainers.txt people are not the decision makers. The only decision makers are Dries and webchick, and Drupal is simply too big to have only two decision makers that have to understand every line of code. That process does not scale. We need to either explicitly give decision making ability to Maintainers (which in turn means Maintainers need to do a much better job of coordinating than we do now) or add and organize the committers so that they actually have bandwidth to deal with Maintainers and all the issues that are outstanding.

Creating a "team of reviewers" means some sort of formality, too. That would also help, but doesn't solve the underlying problem of trying to force too much work through too few people to get it into CVS.

#9 decision makers

look, if your decision is wrong, your solution will be rejected by reviewers or rolled back anyway, being a core committer or not. So if you know the best solution then do not wait, make the call. Even if turns out it is not the best solution it still can't be rejected if it is better than the previous state and nobody is trying to work on better solutions.

#10 Big change stuff

When you deal with smaller changes you can throw the time at a solution and if it gets rejected it gets rejected. That's fine.

But, with bigger decisions you really have the process... proposed architecture -> refined -> blessed -> code it -> reviews and updates -> committed.

If I'm going to code something and the time it takes to get it in will go over 50 hours than I want approval on the architectural concept first. Especially, if there is more than one equal or close to equal competing one. Otherwise I'll blow a lot of time on something that's not going to go anywhere.

If the d7ux stuff wasn't bless up front, the fields in core wasn't architected, or the DBTNG didn't have an architectural blessing initially they would not have gone anywhere. Most things aren't the size of these but somewhere between small and large. Many of these in the middle need this architectural review.

The stuff in the middle is hundreds of man hours to code and work through rather than the thousands these big ones are. But, even at hundreds of man hours there is a lot of time to be wasted and passion to be squashed by dashed hopes. We need to address this. It's a problem. A big enough problem that you'll see some big named contributors who have either stopped or drastically scaled back core work because of.

I don't point out these issues lightly. This is a problem that's been building for some time. It's not just my own thoughts. This started with people smarter and more involved than me.

Plus, the Drupal process is always changing, only servers a small group of people, and is far from perfect (nothing in this universe is perfect). We use to have 3 core committers. Now we have 2 per release. Drupal used to be smaller and simpler. Now we have more than twice the hooks than the last version and several new subsystems. Drupal and the processes around it are changing. Let's make them the best they can be for where we are going.

#11 still no

It's not true. Most of the Drupal 7 changes came from contributors who never waited for any kind of blessing.. And DBTNG was absolutely a decision from Larry and chx, then others jumped in. But Dries was not there.. I think that was the largest patch for D7, it was not on any roadmap for D7, it has no blessing from Dries, so I do not know what else do you want to prove me. It was refactored many times, but that was also Larry's idea for improvements.

http://drupal.org/node/134580
http://drupal.org/node/225450
search for Dries and read his comments.. he do not weight in more than a simple reviewer.

I had two issues which were committed to D7 even the core committer was against it first, and another issue where a change requested by a core committer was rolled back.. If you know what you are doing, do not wait for anything. You won't get any approval for any kind of change upfront, and absolutely not from a core committer.. Only changes requested by the community were blessed by Dries, and as you can see some of those requests were not implemented. http://buytaert.net/starting-to-work-on-drupal-7

If you could tell me one developer who stopped working on core because they had problems with the number of core committers I would be really surprised. Many people stopped because getting something into Drupal core is not as easy as it was some years ago. And the main problem is that issues don't get the required attention, there are too many issues and less and less reviewers. I even made some dull jokes about this on the development list. The process never stalled in the RTBC queue..
If you want to make a change in Drupal you need to be a good reviewer, check this list for example: http://robloach.net/node/91

#12 More Than Issues

A few quick things....

First, I'm not going to list the names of people whom I've talked to in private who are frustrated with the current system. That would not be appropriate but the list is long. And, it's not about the number of core committers but other factors that adding another core committer and having a more varied expertise can help.

Instead of fighting this idea so hard I invite you to start asking more questions and exploring what change could look like. What I proposed does not fundamentally alter core development. It leaves it just as open as it is now but allows it to scale a little more.

Second, not all the conversations about issues and changes happen in the issues. Most would like them to. More of the other conversations should be documented there for the benefit of others. I can't and shouldn't point to issues for everything.

For example, why drupal_add_library() is the way it is in drupal 7 was best discussed in an irc conversation in #drupal.

I understand the need for more reviewers. Especially reviewers with the technical understanding to do a good job. With Drupals increased complexity it's becoming harder to be a good reviewer because you don't know the implications to the system as a whole or the code is, well, complex. I completely agree that we need more reviewers. I even gave a talk on that at a recent drupalcamp. No argument.

But, there is more that can help us process wise than just adding more reviewers. You seem to be adamantly against some of the process changes. Some of which would restore things that were good in Drupal in the past... like another core committer with front end expertise. Drupal 7 was the first version in years that didn't have that.

#13 Get your facts straight

Pasqualle, sorry, you are 100% absolutely factually wrong.

DBTNG traces its origins back to DrupalCon Sunnyvale (March 2007). At DrupalCon Sunnyvale, I sat on the floor of one of the lounges in the hotel and discussed PDO, as a general concept, with Dries. He blessed the idea of experimenting more with PDO as a way to encourage PHP 5 adoption. The next day at the Sunnyvale code sprint I started the code which would, years later, become DBTNG, while sitting next to Dries and asking him questions about it repeatedly. He went so far as to say "Yeah, we probably should move [completely] to PDO eventually."

Over the next few months, he watched the work I was doing closely. When I switched gears to PDO as a full DB rewrite, Dries was actively involved in the blog thread (and others) where I laid out the concept. He went so far as to say "Overall, I'm in favor of the proposed design. Going with PDO seems like a natural evolution -- it's the logical next step for Drupal." If that's not blessing the process I don't know what is. And that's before any of the really major code was written. (chx's main involvement wasn't until February 2008.)

What led to that blog post, too, was Dries telling me in a direct email that he wanted me to "take point" on the database overhauls that I'd been talking about.

When the third draft of the API model was released shortly before DrupalCon Boston, Dries told me that he liked the overall direction. It was still six months and over 100 hours of work before the first patch was committed, though.

DBTNG was Dries-endorsed and blessed from day 0, and day 1, and day 2, and day 5, and day 17. So your assertion that "Dries was not there" and "it was not on any roadmap for D7" and it "had no blessing from Dries" is about as far from the truth as it is physically possible to get.

Please check your facts before making foolish assertions. You're just spreading FUD.

#14 ok

Larry, reading your comments I still see that it was your idea and you made the decisions. And you had no guarantee that your changes will ever be committed.. It was you, who convinced him and everyone else.
If it is all the blessing you need to make big changes in Drupal, then alright let's have more committers who will agree with you till you actually prove it that you are right..

#15 Missing the point

Pasqualle, I think you are missing the point. Larry got concept/design/architecture feedback before he wrote hours and hours worth of code. This is something that not only should happen but does happen. Drupal used to have 3 core committers to a branch and now has 2. Drupal is much more complex than it used to be (e.g., drupal 7 has something like 3 times as many hooks as drupal 6).

There is simply less core committer to go around to give architecture/concept/design feedback.

Good solutions need design and architecture. Especially those that make much of a change or are of any size. The design and architecture part of the process needs more core committer time. This is really not an item under debate. This is an observation of what's going on.

Maybe you have not been involved in any of these discussions or see what is going on. But, this observation has been made by a lot of very active core contributors with long histories. Very smart people. Stop debating to keep things they way they are. The Drupal ecosystem has changes as well as the core dev process. Change is inevitable and can be a very good thing.

To be honest, you are the only one who has expressed negativity for this. More than anything people have contacted me about this being good, healthy, and greatly needed. Before I posted this I had talked to a number of the biggest names in Drupal and they ALL agreed.

#16 the only one

I still do not get it what kind of feedback do you need from a core committer? I always asked the right people when I needed feedback.. I do not need a core committer to get my core issues fixed..

I heard this request quite many times in the last years, but the conclusion always was, that the issue progress stops at review stage. And you also asks for someone (a core committer) who will review your issues.

I started to quote from major core contributors, who does not agree with your request, but you should probably just read the whole discussion:
http://lists.drupal.org/pipermail/development/2009-April/032523.html

I like this explanation:
"Any additional committers need to be skilled willing volunteers with time. ... If you are seriously interested in filling a new position, you should already be doing what you want to do, then we remove a step from your process."

#17 how to rate a patch, a ...

Pasqualle,

and especially Matt, a thousand thanks for discussing this topic.

The main problem is how can i tell the good from the better patch apart? I can't and may other can't too. I can tell (perhaps) which patch or development direction serves _me_ - but will it also suit your needs - how could i know?

And what are the criteria for the core committers to decide - i don't know?

I want to avoid that work / code is thrown in the bin, because of missing communication upfront.

And the process has produced much frustration in the past - eg look at http://drupal.org/node/8 . Ask other developers who have code in the issue queue.

Ask reviewers by what criteria they decice the patch is a good one - i'm interested in the answers.

hope we can find a better process to decide, my suggestion is splitting up drupal in a lot of subsystems (eg. along top level objects/ modules) a maintainer / committer for each subsystem and the decisions of open issues to be made in dated chats / bofs, where all interested persons can join and give there opinion - either could the maintainer / commiter decide or the whole group creates a consensus about the (actual) best way to go.

best
Thomas Zahreddin

#18 Huge change

I've discussed the idea of splitting Drupal up by its major subsystems with a number of people. This would be a huge change to the way we do things and cause a lot of pain. For example, there would be political pain over who the top level maintainers are of a subsystem.

Plus, the core committers typically don't write the code that goes into Drupal. So, if you put a database maintainer in change of that subsystem do they not write code for it anymore and just review other people code? That would be hard. But, if you change to have them lead with their own code you have a second huge change.

The current hierarchy of Dries -> co-committers -> everyone else is something a lot of people like. It means if you have a new great idea you can bring it here and we will try to take a look. It leaves most of the community very flat which many people love.

While I do like setups like what you suggest I think that a cultural change to go to something like that would be too much for the community at this point.

#19 I'm not a core developer, but

I'm not a core developer, but i monitor Drupal's issue queue almost everyday and participate whenever i can.
I saw many long discussions about important or non important features, issues but without any actual result (sometimes even with very good patches). and in most of this threads i can barely see any core maintainer comment or feedback!

Core maintainers should be more active in discussions because they're the gate keepers. people should be notified when they're heading to a dead end.

So i think Matt is correct, Drupal needs more core commiters , or at least some branch maintainers with access to different branches, so core commiters can be only involved in big changes. I think Linux Kernel has something like that. as long as i know one of things that Linux does much is merging :)

For example for new database layer, there can be a different feature branch, people can submit their patches into that branch via branch maintainer. then branch maintainers can ask core commiters to review and merge the branch into the head once it was ready.
If there was a conflict branch maintainers are responsible to make it compatible with core commiter's guide. Changing a working and polished code which is already committed to an uptodate branch is much faster and much easier than a too many non very well tested and not commited patches.

This way many people can become involved without risk of loosing software quality.

#20 For sure the infrastructure

For sure the infrastructure (CVS and the issue queue) doesn't help newbies to get involved and it's really hard to grasp. It's just a cronological list of post. There's no way (besides tags) to group different issues into features, link to releated blocking issues. IMHO it's a really poor tool for project management.
And not having a road map sounds really democratic but sometimes I feel like there's a roadmap but it's not shared with the community. I still don't get who decided to postpone the code freeze and promote some features to high-impact exceptions . And why this crucial decision hasn't been published in the Drupal frontpage instead of personal blogs?

#21 your wish is granted

I do not know how Drupal development could be even more open, maybe you can share some ideas..

#22 Could be more open

Drupal could be a lot more open. If we just threw in every patch that came along. Even if it was just the patches that were up to par for core. Right now that doesn't happen. Many perfectly good patches are turned away. Some live on as contrib modules and that's a good thing.

Drupal core development is not entirely open. The core committers are code and idea gatekeepers. The code gatekeepers we easily see. But, they are more importantly idea gatekeepers. This is a good thing most of the time.

But, if you open Drupal up to everything you'll let a lot of crap in. Being more open is not a better thing.

Project management, especially good project management, is rare and requires some controls. Think of it this way. Someone with kids and a house next to a busy street puts up a fence between their yard and the street. This fence is a boundary and the kids cannot do anything they want. But, that same barrier is there to protect and guide them.

What I suggest in #2 and #3 above is a little project management. It's understanding your users and communicating with them effectively. Currently, we only effectively communicate with people who are good and using issue queues. This is a small segment of users. With other users we need some improvement.

#23 communication

I mean open in a way, that every info about the project is freely available, you can see every movement of the Drupal project. Not open in a way, that you can do whatever you want..

I do not know what should be done better for this. Should someone categorize the issue queue and write about the progress of interesting issues? Or have a place on d.o where the unstable releases are announced? so at least after the issues are fixed, you see a short list of important commits on d.o instead of the mailing list.. Or should we create an important list of blog posts from Drupal planet?

Yes, if you want to see the progress you need to follow the different sources. If someone think he/she can communicate the project management better, then go an help. You can do it..

#24 By being more open (and I

By being more open (and I know I went off topic from Matt post) I mean two things.
1. As Pasqualle said it would be great to have someone categorize issue queues, summarize progress, and post important events on d.o front page (like the code freeze delay I mentioned before that Dries kindly posted after a comment on his blog)

2. As I said before, sometimes it's not really clear who take some decisions (like the one I mentioned about the delay and the promotion of certain features to release blocker). We should decide if code freeze is dependent on features (and in case someone should decide which are these features - i.e. - they make polls at wordpress) or on a time-based deadline (code freeze is day xyz whatever features are ready).
For what it worth I 'd say let's work by feature freeze (as stated here) and find a way to make the selection of to be implemented features open to the community (we almost did it, there was a survey if you remember), and when possible, as Matt wrote, try to find a common ground on the architecture/direction to take even before writing any code. Code is gold but even really expensive in terms of time and rare. Let's find a way to reduce waste.

#25 I decided

I would really like to encourage you and everyone to take a bigger role in the Drupal community. I am almost sure you won't find a more open development community. Do not wait for someone else, take the lead, communicate your ideas, improve and make everything better.

If you compare Drupal to democracy. Who made the decision? you did..
It was simply a bad idea to have a code freeze on the same day as DrupalCon, so it was fixed quickly without any resistance..
I think I can say that we do not work with deadlines on Drupal, we just try to make it better for us and for everyone.. That could be the main motivation why there are some exceptions, why Dries try to get some extra attention for those features, but that still does not mean that it will be actually implemented, as someone have to do it. Someone have to make the decision to work on it, and not just talk about it or wait for someone else..

#26 I think I was a little too

I think I was a little too rough at the beginning.
I'm aware that Drupal development is open (and probably more democratic than our democracy,at least here in Italy! AhAh ). Anyway I'm just trying to support the idea that we should try to improve the issue queue and the project management in general to include more people in the development (by making it easier to follow development) and maybe revise our development habits by taking some decisions (features to develop and architecture directions) in advance. But I'm just 'thinking out loud' and I'm sorry if it might sound like I was accusing someone.
And yes it's time to for me to participate more.