RFC: Start development in one language

[thewilkybarkid]

This is my personal suggestion as a community member, and not directly from eLife.

Background

• eLife built their microservices using both PHP and Python. The division was based purely on who was writing each service.
• For Libero, eLife proposed realigning language use so that PHP is for browser-facing services, and Python for APIs.
• There’s no requirement to use the Libero code, it’s specification-first so Libero-namespaced code doesn’t have to be used (Hindawi have mentioned using PHP/Drupal for APIs and JavaScript for browser-facing services).
• Long-term Libero should at least provide adapters (client libraries) for different languages.
• One of the reasons for using microservices is the ability to (responsibly) use different languages. For eLife, and the known potential uses of Libero, there is no specific reason to use PHP or Python for a particular service: neither holds clear benefits over the other.
• eLife doesn’t currently have an in-house Python specialist (currently hiring); the team overall has more PHP knowledge.
• Choosing any language to develop in is opinionated: developers/groups outside eLife want to get involved/use Libero won’t necessarily know either language.
• PHP is well known in the publishing world due to OJS and Drupal usage.
• A small amount of code was written in both languages during the recent sprint.

Problems

• Starting Libero in two languages will see a lot of duplication of code (eg API client, Event Bus client, logging, coding standards).
• Current lack of Python specialism in the eLife office.
• For a group who currently don’t use either language (eg RSC and Hindawi), it’s two to learn (if not rewriting).

Suggestions

• Solely use PHP for Libero-namespaced code until at least the MVP is released.
• Look at creating adapters in other languages (starting with Python) afterwards.
• Be willing to use other languages when there is a suitable reason.

Concerns

• Might accidentally create a monoculture, or at an impression of one.
• Harder for the wider eLife team to contribute.
• Unknown impact outside of eLife (might block contributions from some areas, though could help in others).
• PHP has an image problem, though this isn’t justified with ‘modern’ PHP.

[girishmuraly]

Modern PHP as you say has made progress in the last 2-3 years. Frameworks like Slim and Symfony make it equally competitive with other languages for creating microservices. As you said, if we keep it specification-first, this should help.

On the one hand it means non-PHP developers will have starting trouble with the respective services but on the other hand, those with PHP knowledge can work on any part of Libero as needed.

Hindawi at the moment does not have developers but are going to hire by end of the year. Keeping it in PHP means it is slightly more easier to hire and cheaper. Also for us, we will be exploring creating the journal CMS in Drupal (PHP framework) so it gives us more surface area for our developers to work on.

[pmollahan]

At Digirati we agree it is difficult to remain objective in the choice of development language, particularly in context of developing a solution that is aimed at wider adoption by other publishers and service providers (both in context of extending and enhancing the platform for further custom implementations and to provide a hosted service).

There is no right or wrong answer in this case. So the following are some thoughts / considerations:

(a) Retaining Python

Re Background:

Going back to Digirati’s original involvement with eLife in 2015, the selection of Python for the back end services that compromised the “bot” reflected the data processing aspect of the work required and some library / language constructs that fitted well with the emerging AWS based architecture. There was also an extensive pre-existing Python codebase at eLife so the opportunity to build upon that was seen as a means to maximise value and ensure faster delivery.

The introduction of the PHP microservices architecture as part of eLife 2.0 has clearly worked well, and although these PHP microservices were “backend services” the split between the Python and PHP elements were broadly well defined with the “bot” remaining largely as it was.

The original decision for the new Libero platform to use PHP for front end and Python for the wider backend services seemed to be a sensible split. As discussed at various points over the 2 day workshop at the start of the community sprint there were no major concerns raised although there was some discussion about the potential for using other technologies on the backend (Javascript and / or PHP).

Our assumption was that this strategic decision (i.e. Python for backend, PHP for front end) reflected:

• The split of the work that had been completed during the development of the walking skeleton, and the suitability of the languages used for the various hypotheses
• The already existing Python codebase that could be (in some cases) lifted and shifted for reuse in Libero
• Consolidation of the back end services into a single language

From our perspective, we deliver implementations in PHP and Python (as well as Java, Javascript etc.) and the front end / back end split resembles other platforms that we have been involved in. So having multiple languages within a “single platform” although challenging from a resource perspective often reflects the domain and platform complexity.

(b) Delivering in PHP only

All of the points made seem sensible, the key point being resource availability for the eLife team to go ahead and deliver the MVP.

However, if the decision is based on current resource availability and is only for the MVP should a separate RFC be created to determine the longer term strategic decision? Or would this RFC defer back to RFC #4 at that point?

Whilst adapters for other languages may be developed post MVP; unless the platform technical strategy suggests that these should be considered core, one would assume they would remain an implementation of the platform.

While the argument to have one language only is clearly a benefit for any publisher or service provider wishing to use and develop on the platform from a support and maintenance perspective, the potential split of domains between front end (the delivery website, front end patterns, any CMS etc.) and back end (ingest and data processing, transformation, workflow, integration etc.) may mean different resources are required to be involved irrespective of the (single) language chosen.

What’s difficult to quantify and is clearly hard to be objective about is whether PHP is the best single language to deliver this. With the community dependent on the delivery of the MVP for progress, then the decision can only really be determined by eLife as the team working on implementation.

(c) Reaching a strategic decision for the community

When faced with the challenge of helping assess and select the “appropriate” language for an implementation we have typically looked at a series of factors that apply to the client for whom the implementation is being developed.

• General language requirements. We assess at a high level the types of domain challenges that exist and availability of libraries to support these. Based on the details above it seems that in this instance the assumption is that both Python and PHP are equally suited.

• Language alignment for the ecosystem or wider platform. This will be context specific (and usually it's only client specific) but in the case of a community publishing platform, integration with other systems in the wider ecosystem such as peer review or other tools should be considered. Again this is a context specific one - but perhaps with the work that a number of publishers are involved in with Coko that could / should influence here too.

• Integration with required services. This would include areas like message bus, workflow systems, cloud infrastructure services etc. Other considerations might include the reliability of“long-running” processes etc.

• Existing staff skills - with the community only developing this is really only eLife initially with consideration for Hindawi, other publishers, service providers including Digirati

• Availability of developers with those skills (this would need canvassing across the community as it develops and grows)

[thewilkybarkid]

One possibility for the future is to have multiple Libero-namespaced implementations. For example, if the initial content store was written in PHP and kept reasonably simple (that could cover a lot of use cases, but doesn't have the complexity of Airflow), a more complicate version could be released in the future that does more but is more complex to run (eg could be in Python and based around Airflow). They both implement the same API, just in different ways (which fits the one-size-doesn't-fit-all approach).

[thewilkybarkid]

Going to rename the RFC to be a bit more accurate.

[BlueReZZ]

As a general point of information about PHP in open source: we've been talking to people experienced in open source recently which has led to conversations with other service providers too. We found that the Learning Management Systems space is abundant with both open source applications and service providers offering hosting, bespoke development and maintenance. The two main OSS platforms (Moodle and ATutor) are written in PHP with others in Java. Service providers we've spoken to therefore offer PHP and Java development, although all would offer to host anything that is cloud native and uses containers, so the development language is less important to some.

[giorgiosironi]

Adding my 2 cents to the good arguments being put forth here.

A polyglot approach needs to avoid accidental lock in into a language, for example it's easy to adopt a data format or technology that is only suited (or mostly suited) to a single language when that language is the only option. So far we have seen some cases:

• Twig for the patterns-core, PHP-specific
• RelaxNG, well supported in multiple languages
• Airflow for (the future of) content-store, Python-specific
• Static analysis, each language has its own tools to setup and configure

For an MVP I am not concerned about this problem, so I am not opposed to using a single language for that. Medium-term it is problematic to only work in one language if the intent to support multiple ones is present (with 2 as the minimum).

Some concern for the already existing projects created during the Sprint, although some decisions can be ported over (which database to use) most have to be re-evaluated but probably copied over from existing PHP projects.

[BlueReZZ]

This matter can be closed since there were no major objections and work has now started in PHP by eLife with a plan from Hindawi to develop in PHP soon too. There have been some good opinions recorded here so thank you for the contributions - a good future reference for the community.