[go: up one dir, main page]

Page MenuHomePhabricator

Make "Quick" MW install a thing
Closed, ResolvedPublic

Description

Objective

  • Someone with basic coding experience gets a MW install in less than 3 steps and under 5 minutes.
  • Someone with little to no prior setup, gets a MW install in less than 5 steps and under 15 minutes.

Background

TLDR: no docker, no apache, no mysql.

For new MediaWiki contributors who have some prior experience with software, the experience should feel like:

  • clone repo
  • start server

For people with no recent software dev experience and/or with pristine workstations, the more honest description would be:

  • Install git, php, composer
    • Linux: apt-get install git php composer
    • Mac: Install Homebrew, then brew install php composer (git is pre-installed).
  • git clone mediawiki/core mediawiki, git clone mediawiki/skins/Vector mediawiki/skins/Vector
  • start server

The key realization here is that php comes with a built-in web server, and has for many years. We actually used to run WMF CI this way for several years. All you need to get a working web server is php -S localhost:4000 in a directory and you're up. By avoiding Apache you remove the knowledge and hassle with configuring a general web server on your laptop and "adding" your MediaWiki site to it (and potentially conflicting with other things), as well as the way it can feel intimidating to keep that service running correctly. Useful skills to acquire after you've been developing software for a few years, but close to zero value during your first year and mainly serves to get in the way and distract things.

The other part is that MediaWiki supports sqlite, so no MySQL needed. This is, perhaps surprisingly, actually how the current iteration of mediawiki-docker runs as well. That's probably half the benefit of using a container-based dev env, but we don't even tap into that potential today. In any event, all that to say, MediaWiki has first-class and documented support for sqlite , is continuously tested with this on every patch, and has been for over a decade.

Lastly, by not involving Docker, we reap several benefits:

  • Time: It easily takes up to hour to initially get all the containers and image layers downloaded.
  • Complexity: downloading, installing, configuring a global deamon. And managing it every day to keep it running correctly.
  • Complexity: none of our Docker images support ARM processors.
  • Complexity: you sometimes have to "enter" the container to run certain commands, but then leave the container to run other commands.
  • Performance: Especially on Mac or Windows, the containers run in one or two layers of emulation and thus aren't particularly fast. It's not very different from the Vagrant days, except with Vagrant we didn't pretend it was a "fast" container instead of a full VM. We've just hidden the VM inside Docker Desktop now.
  • Principle value: for most people the simplest solution would involve installing Docker Desktop, which is proprietary software that promotes a freemium/premium subscription based service.

The "time" factor alone disqualifies MediaWiki-Docker from a homework-free solution in which you can get someone with basic coding experience to have a MW install in less than 3 steps in under 5 minutes.

How

In a nut shell:

  • php maintenance/install.php …options
  • php -S localhost:8080

The first one is currently preset in composer.json already as composer mw-install:sqlite. The second as composer serve.

Prior art at https://wikitech.wikimedia.org/wiki/Performance/Fresnel#Quick_MediaWiki.

Scope

  • Verify the above is accessible.
  • Evaluate whether potential conflicts with an existing LocalSettings.php is a likely hurdle. If so, write a patch to make use of MW_CONFIG_FILE to e.g. give it LocalSettings.quick.php instead and fix install.php to write to this instead the default location.
  • Document this somewhere and test-drive it during the virtual wmf-staff hackathon later this year.

Event Timeline

In the longer term it would be great to have a single entry point to installing MediaWiki and other management tasks (installing an extension, adding wikis to a wiki farm etc), similar to e.g. drush / drupal-console or symfony-cli. In part because the same command with different options could then be used for bare, Docker, k8s etc, which would simplify command memorization and documentation a lot.

In the longer term it would be great to have a single entry point to installing MediaWiki and other management tasks (installing an extension, adding wikis to a wiki farm etc), similar to e.g. drush / drupal-console or symfony-cli. In part because the same command with different options could then be used for bare, Docker, k8s etc, which would simplify command memorization and documentation a lot.

That would be nice, indeed. Linking T333668: Shared tools for common development environment tasks as a place to continue discussion on that.

Someone with basic coding experience gets a MW install in less than 3 steps and under 5 minutes.
Someone with little to no prior setup, gets a MW install in less than 5 steps and under 15 minutes.

Please consider the https://gitlab.wikimedia.org/kharlan/mediawiki-metal MediaWiki-Metal project from the Athens Hackathon, it would be nice to build on that if it meets the requirements for what you have in mind. Demo: https://asciinema.org/a/Z6AG7SQgxA3kPMWNWnAR7s1bM

Using that, you can have a setup with PHP built-in server and SQLite using 3 steps (clone repo, composer install, mw-metal setup). If you want to use Apache and MySQL, it is one more step to use brew install httpd mysql or equivalent apt commands on Debian. (For my 2 cents, it is a bit of a disservice to leave new developers with SQLite, which doesn't work well with some extensions, and built-in server, which won't have the same URL scheme as they're used to seeing in production Wikipedia.)

We actually used to run WMF CI this way for several years. All you need to get a working web server is php -S localhost:4000 in a directory and you're up. By avoiding Apache you remove the knowledge and hassle with configuring a general web server on your laptop and "adding" your MediaWiki site to it (and potentially conflicting with other things), as well as the way it can feel intimidating to keep that service running correctly. U

It didn't work with Parsoid and VisualEditor in PHP 7.4 and lower, though. This doesn't seem to be an issue anymore in PHP 8.1 but I haven't investigated if that has more to do with how we reorganized Parsoid code in core. ~2 years back, the issue is that the same thread which received an API request to make a VisualEditor edit would attempt an internal API request to Parsoid, and this all happened on the same thread so the process would hang. Anyway, it should be fine now, but it's one reason to be wary of relying on PHP built-in server for our development environment.

If this is partly for ease of installation by first-time patch submitters, we should bear in mind that the new developer also has to jump through the gerrit setup and wikitech account creation hoops, and setting up an ssh key etc. If one already has these things, then we can indeed get the install time down to something very short.

@Krinkle You mentioned last week that this was mostly a documentation task to be done by tech writers. Is that right? Any development or actions required from our side?

cc @Bmueller

I understand running docker on Mac and Windows machines can introduce unwanted complexity and speed problems. And this seems like it will be an alternative to folks who have had problems with docker. I think linking to MediaWiki-Docker wherever this documentation ends up would be nice since it seems to achieve the two objectives of this task for folks who've not experienced issues with docker—it's a fast way to get started.


Time: It easily takes up to hour to initially get all the containers and image layers downloaded.

As a data point, following the instructions in DEVELOPERS.md docs on a clean debian VM with 1GB ram and 1vCPU took 8 minutes to get a running MediaWiki.

That was after install of git, docker-compose, and cloning core. Cloning the whole of core was the slowest part of all this for this under-powered machine (took 5 minutes, mostly due to a slow disk speed on this machine).

Cloning mediawiki/core and mediawiki/skins/Vector, running composer update, and running php -S localhost:4000 does indeed get a MediaWiki running in about 15-20 minutes, with bulk of the time spent in cloning the repos.

Too bad I didn't know this when I was getting started with MediaWiki. (It took me months to eventually get my docker-based install to work.)

Couple of pain points that slow down the process:

  • Git cloning needs to be done from SSH URI which has LDAP username in it. Not sure why we have the username here. (On GitHub, SSH URI is same for everyone – git@github.com.) This necessitates creation of a developer account *first*. Maybe something that could be streamlined with Gitlab migration.
  • LocalSettings.php is generated automatically based on prompts given in the web installer. Great. But it prompts to download the file which is not great UX as browser sends it to the Downloads directory and user needs to copy-paste it over. This requires reading the instructions. Why can't MW directly put it in the installation dir?

Git cloning needs to be done from SSH URI which has LDAP username in it. Not sure why we have the username here. […] This necessitates creation of a developer account *first*.

The username is not required for cloning. The initial clone can happen over https:// to save time from finding the right SSH address. This is what the documentation recommends today as well.

If or when you do use SSH, the username is also not required. Git will call the ssh program internally, and that in turn has its own default and configuration. The default is to use the same username as your personal Terminal. It's possible that this is the same for you as what you picked for the Wikimedia Developer account. If not, there are several ways to make SSH remember this. For example, when people with Toolforge or Wikimedia Production access use SSH, they tend to use ssh something.wmcloud and ssh something.wmnet, and that does not require a username either. This can be configured in ~/.ssh/sshconfig (Example):

Host gerrit.wikimedia.org
    User krinkle

Or, my personal favourite, I do it via ~/gitconfig as follows:

[url "ssh://krinkle@gerrit.wikimedia.org:29418/"]
	insteadOf = https://gerrit.wikimedia.org/r/

This means two things:

  1. If you git clone with the default URL that we place in documentation anonymous HTTPS clone, it will automatically use SSH instead.
  2. If you already have existing clones, it will automatically perform future actions (git-review, git pull etc) over SSH instead. So you do not need to change any git remote.

In short, I would recommend, when getting started:

  • Clone over HTTPS.
  • Create a Developer account (if not already), or set up .gitconfig (if not already).

The outcome is that if you already did gitconfig, it will use SSH. If it didn't do it before, it will clone with HTTPS and then later automatically become an SSH clone to easily submit patches with.

Not sure why we have the username here. (On GitHub, SSH URI is same for everyone – git@github.com.) […] Maybe something that could be streamlined with Gitlab migration.

The username part is a bit of a distraction here. In order to have a username, you need to have an SSH account. If you already created an SSH key, then you likely also have the SSH config that goes along with it already, which means you can then use ssh:// without specifying any username. Your ssh config will know which key and username to use. This is the same number of steps as with GitHub, except that when setting up the SSH config with github you don't specify the username in sshconfig.

The outcome is the same either way, you git clone with a generic URL - no username.

The reason we have usernames is that we use SSH not only for Gerrit, but also for Toolforge and Wikimedia Production.

This comment was removed by Krinkle.

Hi everyone, the quickstart workflow is now documented at Local development quickstart. The goal of that page is to provide the shortest, simplest path to running MediaWiki locally for people who don't want to use Docker. Feedback and edits welcome! For feedback about MediaWiki development environments in general, I've started a comparison of available options on Project:Development environments.

As a first step to integrating the quickstart workflow with the MediaWiki development docs, I'd like to replace the Local installation section on the How to become a MediaWiki hacker page with a link to Local development quickstart.

  • Someone with basic coding experience gets a MW install in less than 3 steps and under 5 minutes.
  • Someone with little to no prior setup, gets a MW install in less than 5 steps and under 15 minutes.

If we exclude installing git and installing or updating a package manager, the quickstart process requires 5 steps and takes about 6-7 minutes on Windows, Mac, and Linux. The most time-consuming step is cloning MediaWiki core, which can take several minutes depending on connection speed. For more information about how I measured setup time, see T348899.

Change 977124 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] mw-install:sqlite: Use run.php entrypoint

https://gerrit.wikimedia.org/r/977124

Change 977127 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] composer:serve: Disable process timeout

https://gerrit.wikimedia.org/r/977127

Change 977128 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] composer serve: Make use of dev router

https://gerrit.wikimedia.org/r/977128

Change 977136 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] mw-install:sqlite: Load DevelopmentSettings

https://gerrit.wikimedia.org/r/977136

Change 977137 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] composer: Consolidate local dev commands in a namespace

https://gerrit.wikimedia.org/r/977137

Change 977211 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] composer serve: Set log directory

https://gerrit.wikimedia.org/r/977211

Change 977127 merged by jenkins-bot:

[mediawiki/core@master] composer serve: Disable process timeout

https://gerrit.wikimedia.org/r/977127

Change 977137 abandoned by Kosta Harlan:

[mediawiki/core@master] composer: Consolidate local dev commands in a namespace

Reason:

https://gerrit.wikimedia.org/r/977137

Change 977136 merged by jenkins-bot:

[mediawiki/core@master] mw-install:sqlite: Load DevelopmentSettings

https://gerrit.wikimedia.org/r/977136

Change 977128 abandoned by Kosta Harlan:

[mediawiki/core@master] composer serve: Make use of dev router

Reason:

https://gerrit.wikimedia.org/r/977128

Change 977124 merged by jenkins-bot:

[mediawiki/core@master] mw-install:sqlite: Use run.php entrypoint

https://gerrit.wikimedia.org/r/977124

Change 984225 had a related patch set uploaded (by TheDJ; author: Krinkle):

[mediawiki/core@master] build: Set PHP_CLI_SERVER_WORKERS=8 in composer serve

https://gerrit.wikimedia.org/r/984225

Change 984225 merged by jenkins-bot:

[mediawiki/core@master] build: Set PHP_CLI_SERVER_WORKERS=8 in composer serve

https://gerrit.wikimedia.org/r/984225

Change 984798 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] LegacyLogger: Emit to stderr if MW_LOG_STDERR is set

https://gerrit.wikimedia.org/r/984798

Change 977211 merged by jenkins-bot:

[mediawiki/core@master] composer serve: create and set log directory

https://gerrit.wikimedia.org/r/977211

Change 984798 merged by jenkins-bot:

[mediawiki/core@master] LegacyLogger: Emit to stderr if MW_LOG_STDERR is set

https://gerrit.wikimedia.org/r/984798

Krinkle claimed this task.
Krinkle moved this task from Blocked/waiting to Current Sprint on the MediaWiki-Platform-Team board.