This page is obsolete. It is being retained for archival purposes. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on the information here being up-to-date.

The German version is available here: Tool Labs/Roadmap de

The Toolserver is an infrastructure that hosts software supporting Wikipedia and its sister projects. Our expectations of such an infrastructure are the following:

• Any volunteer can request an account to develop tools to support, improve or analyze Wikimedia projects.
• It makes sharing of code and knowledge as easy as possible.
• The infrastructure is well maintained and available. Technical failures should be an exception.
• The tools developed in such an infrastructure are running in an environment that is similar to the productive Wikimedia projects. The data of all Wikimedia projects and OpenStreetMap are easily queriable. By data, we mean the subset that is visible to normal, unprivileged registered users.
• It offers user databases and shared storage for projects.
• Tool maintainers can take part in the definition of features and software needed in such an infrastructures.
• Ideally, it offers a staging process for software that can be used in production.
• There are admins available, ideally in different timezones, that respond to the tool maintainers’ requests in a reasonable time. They should deal with maintainers with different amounts of technical knowledge and be open and welcoming to newcomers.

• Technical concept: The Toolserver is mainly used by individual developers. Though there is a concept of multi-maintainer projects, this is not considered to be the default. The ideas of sharing software and of offering a staging process are at the core of Labs’ concept though.
• Responsibility: The Toolserver has been maintained by very few people for years. They were only partly paid and essential parts of the work have been done by a few volunteers. This has not only led to an immense work load for them but it has entailed a lack of documentation, too. This makes it more difficult to bring new sysadmins in to better share responsibilities. Labs/Tool Labs have been designed and built by paid WMF staff from the beginning. Volunteer admins are welcome but the responsibility for the project as a whole is assumed by an institution that has got more hands/heads to share the knowledge and more time for documentation. A single person leaving would thus not endanger the whole project.
• Labs' dedication to OSI-approved licenses/free and open source software fits well into Wikimedia’s and Wikipedia’s concept of freeing knowledge. The Toolserver also hosts closed source software on both the level of infrastructure and tools.

WMDE clarifies the following things:

• The Toolserver is a community project. WMDE is in a coordinating role to support it, but the migration itself has to be done by the active community.
• No tool maintainer willing to migrate their software shall be left alone. WMDE will try to assist them as much as possible. All tool maintainers who want support to migrate their tools are asked to tell us what exactly they need until the end of September 2013. (“Us” being either the Toolserver contact persons at WMDE or the WMF ops working on Tool Labs.) After this deadline, WMDE will not be able to take calls for support anymore.
• The above mentioned commitment to free/open source software means though that no tool maintainer can claim the 1:1 transfer of their tools to Labs. It is possible that some tools will have to be ported to open source alternatives by the tool maintainers. WMDE does not accept responsibility for the preservation of all tools and WMDE will not be able to take care of proprietary software running on the Toolserver. According to WMF Legal, it is possible to negotiate exceptions: "If there are particularly important tools impacted by this rule, then on a case-by-case basis we may be willing to take a license to specific software. For example, if a university owns specific software, and are willing to license it to the Wikimedia Foundation and our volunteers (not just one specific volunteer), then we could discuss that. But the licenses would be very broad, and we would not be willing to negotiate them at great length or detail."
• WMDE has run the Toolserver, WMDE is coordinating the transition process. We are not responsible for Tool Labs though - that is a WMF project.

• Technical support for migration: Our software developer Johannes Kroll can be asked for help if tools need adaptations to run in Tool Labs. He will accept support requests until September 30th, 2013, he will help to estimate the amount of programming work needed. If desired, he can help to modify tools. A part of his working time is dedicated to this between October and December 2013. This is why he will prioritize, depending on the number of requests.
• documentation of the migration process from Toolserver to Tool Labs

Tool Labs consists of two projects inside Wikimedia Labs: "Tools" and "Tools-beta".

"Tools" is a stable production environment for tools. This project is built and maintained by three administrators, two of them being WMF staff and one a volunteer. More volunteers can be added, if they are ready to sign an NDA with WMF (like on the Toolserver). Root access is quite restricted here to keep the environment itself stable.

"Tools-Beta" is a more experimental environment for tools. This project is community-maintained and more flexible in its architecture so that tool maintainers can experiment with changes in the environment.

This road map clarifies

• when Tool Labs is technically ready to replace the Toolserver,
• which steps have to be undertaken to migrate.

Estimated Time of Completion	Topic	Goal / Milestone	Responsibility
From 2013-05-24 on	Database access	Tool Labs offers database replication to the first testers. There are already volunteers for this.  Done	Community
From 2013-05-24 on	Database access	See how many of the complicated queries could not be solved by federated tables nicely and deal with them separately together with WMF ops.  Done Answer: none found.	WMF, community
From 2013-05-24 on	Migration of tools	RENDER starts to investigate its migration process.  Done	WMDE
2013-05-24/25	Amsterdam Hackathon	Workshop on migrating tools to Tool Labs, hands-on help  Done	WMF, WMDE, community
2013-05-31	Migration of tools (depending on db replication)	E-Mail is sent to tool maintainers asking for volunteers to move first and to document the process.	WMDE, community
2013-06-12	Database access	Replication of database cluster s7 (with Central Auth) is available.  Done	WMF
Between 2013-06-15 and 2013-07-31	Migration of tools	A small number of tool maintainers migrates their tools and serves as example (small and easy ones as well as a bigger and more complex one).  Done	community (supported by WMDE and WMF ops)
2013-06-30	Logs	Anonymization procedures are approved by WMF Legal.  Done	WMF (ops and legal)
30.06.2013? (depending on OSM details, maybe RENDER, too)	General status of Tool Labs	Tool Labs has all necessary features to replace the Toolserver.  Done for 99% of use cases.	All: WMF, WMDE, community
2013-09-30	Migration of tools	Deadline for tool maintainers to ask for technical support for migration	Community
2013-09-30	Migration of tools	RENDER is running in Labs  Done	WMDE
2013-09-30	Accounts	New people interested in a toolserver account are sent to Tool Labs.  Done on TS wiki	WMDE
1st half of October 2013	Clean-up	Mail to all inactive accounts to download their data until Dec 31st 2013. Archiving script will be provided by TS admins.  Done	WMDE
1st half of October 2013	Accounts	All (single maintainer and MMP) accounts are set to expire at the beginning of January 2014. First renewal until June 1st 2014. Second renewal until June 30th 2014.	WMDE
October 2013 to March 2014	Migration of tools	Monthly reminders with information are sent to all tool maintainers to migrate or to backup their data somewhere else.	WMDE
1st half of January 2014	Clean-up	All ts account are set to expire.  Done Almost 300 users reactivated their accounts.	WMDE
still open, details will be discussed in Amsterdam are being rediscussed	OSM	Technical requirements of OSM (Labs part) are defined and in place. In progress Migration is ongoing.	OSM, WMF
exact E.T.A. unknown, ca. end of Feb 2014	OSM	Postgresql/postgis is running on dedicated hardware  Done	WMF
E.T.A. unknown	OSM	Tileserver v.1 runs in its own project inside Wikimedia Labs  Done	OSM@Toolserver Volunteers
Mid-March 2014	Infrastructure reloaded	Wikimedia Labs have moved to equiad datacenter  Done	WMF
Depends on Labs having moved	Database access	wikidatawiki and commonswiki are available on all clusters for federating	WMF
E.T.A. unknown (Labs moved)	Incoming e-mail	Incoming E-Mail function for maintainers and tools is in place. (Outgoing e-mail is working.)  Done bugzilla:58796	WMF (ops and legal)
2014-06-30	Migration of tools	All tools that shall move have moved. What is left on the Toolserver will be archived until August 31st, 2014 and then deleted before the final physical decommissioning.	community (supported by WMF and WMDE)
2014-07-30	Decommissioning	We have a plan for the Toolserver hardware that can still be used.	WMDE and community
2014-07-30	Decommissioning	We have a fixed date for the final (physical) decommissioning of the Toolserver.	WMDE /WMF
2014-08-31	Decommissioning	Deadline for requesting backups, data deletion ahead!	WMDE
unknown / depending on OSM migration	Decommissioning	All data is purged from the Toolserver.	WMDE, TS admins
unknown / depending on OSM migration	Decommissioning	Toolserver is decommissioned and donated to some other cool project	WMDE, TS admins
2014-12-31	Buffer: Four extra months for unpredictable things.	Everything is really, really done.	All: WMF, WMDE, community

• Overview of features needed in Labs/Tool Labs (community): Wikimedia Labs/Toolserver features needed in Tool Labs
• List of Tools (Silke and community, incomplete): Toolserver/List of Tools
• Migration Docs/FAQ (Silke, to be continued): Wikimedia Labs/Migration Of Toolserver Tools
• WMF Engineering goals: Wikimedia Engineering/2012-13 Goals#Wikimedia Labs
• Tool Labs Database Plan: wikitech:ToolLabsDatabasePlan
• List of tools hosted in Tool Labs: https://tools.wmflabs.org/