More automated software systems?

Following up from https://members.webarchitects.coop/t/invoicing-system-invoiceplane-crm-discussion-and-feedback/ (I’m making a new thread as this is a bit higher level discussion).

Me and @chris have together spent some time trying to get ERPNext into a good shape for Webarch, have encountered many roadblocks. It could still be a useful solution, but there are other options, and it would be worth putting the options forward to be discussed at a future committee meeting. Maybe other people on here have some useful to offer too!

There might be a number of goals here, and there is risk of scope creep (as some solutions offer lots of features we could put to use) - but one of the high level goals is to automate as much of the ordering process as possible to provide a smoother service for clients as a more efficient process for workers.

Imagine a future where a client can click, click, click and have a WordPress site up and running all setup with recurring billing, with no need for any manual intervention!

Currently we’re some way away from that. The clients must email, a techie must login to a machine and provision what they want, finance worker must update some documents, invoices need to be created, and each invoice needs to be paid separately (no recurring billing) - please correct me if I got any details wrong! I am not so involved in each of these processes.

Also, each person tends to have their own areas and tools making visibility across the team hard (even with only small number of workers).

The areas/tools we have been looking at are:

  • invoicing / billing
    • recurring billing
    • HMRC VAT API submissions when registered
  • issue tracking
    • currently using RT and mailman lists
    • aim would be to get everyone at least using the same system
  • central client list with contact information, address, etc
  • co-op membership management, shares, etc.
  • easy updating public website
  • “shopping cart” online ordering system with product database
  • timesheets for client billing
  • client portal
    • view products / invoices / support tickets / etc.

This is quite a big list of things to tackle! The idea is not to solve everything, but keep it in mind at least.

ERP

The dream is to have an ERP system that can keep track of all/most of this stuff in a unified way. You would only have to login to one site and have everything available and nicely linked.

We evaluated quite a few open source options and ERPNext came out as the top contender. We have a dev version deployed at dev.erpnext.webarch.co.uk.

However, the main challenge is understanding and configuring it for our needs. Its has a phenomenally large number of modules and a bit overwhelming at times. Quite a few things turned out to be quite fiddly to setup, potentially buggy, and sometimes unintuitive.

Both mine and @chris’s feeling was that we don’t have the resources right now to set it up correctly. An option would be paying the ERPNext people to do it, but that would take quite a bit of time to even specify what is needed, and then pay them, and it seems resources are not plentiful enough for that.

It’s built in a very generic way, which makes it sometimes a bit hard to do seemingly simple things (I was trying to set it up to have an inventory of products, and create a recurring subscription, and manage the invoicing - it sort of managed it, but feels a bit rough still). The issue tracking feature in it totally failed to do what the docs promised (replying to the original message with a ticket number), the recurring billing with stripe didn’t seem to work as described in the docs, and quite a few other little things…

I would not write it off yet, it has a nice clean look, is actively developed, is very customizable (can hide/add fields), has a API, etc… and seems it has the features sufficient for some of the more basic client/member management.

For payment it can connect to PayPal, Stripe, GoCardless, and some others. With various degrees of integration.

One way forward with ERPNext would be to just use it for some of the more basic needs (client list, maybe for managing invoices individually without recurring billing) where it seems plausible, and hope some of the other aspects are possible to make use of when there are some more resources available, or they make them simpler to setup.

Billing / invoicing

If just trying to address this aspect one option would be https://stripe.com/gb/billing - it’s not open source of self hosted, but given at some point we have to use a payment provider maybe that is acceptable?

Me and @chris logged in and could create a simple product listing and a recurring subscription based on it, the invoices got emailed out as expected with a link to setup payment. This solution be for storing the product lists, client billing information, client subscriptions, and invoicing. Recurring billing is handled automatically by stripe charging the clients card each month/year/whatever.

GoCardless is another option for taking payment - based on direct debit mandates. It seems to have less flexible concepts of products, but maybe sufficient. Subscriptions can be changed to some extent afterwards without needing to agree a new direct debit mandate.

Another option is to go further into the non-FOSS SaaS hole and use Xero (quite common for other CoTech coops). It has quite a few useful features like inventory (product) management, will handle invoices, do payroll, time tracking integration, etc. It seems Xero does not have recurring billing subscriptions though, just options to integrate it with another system (e.g. GoCardless).

Personally, I would lean in this direction, given a payment gateway is needed at some point for slick billing (currently PayPal is available), and accept the compromise of non-FOSS software until a time that Webarch has enough resources to integrate a FOSS solution. It’s definitely not clearcut though, and it doesn’t tackle any of the online ordering or provisioning needs.

Online ordering with shopping cart

As mentioned, the above invoicing / billing solutions don’t address online ordering at all.

One of the simpler options would be to setup a WordPress site with a shopping cart plugin (e.g. WooCommerce) - which themselves offer recurring billing solutions (paid extra).

There is someone trying to create a hosting shopping cart on top of WooCommerce! (plugin does not look usable).

Maybe with some custom WordPress plugin it could, in time, be hooked into a automated provisioning system…

Automated Provisioning

My feeling is that at some point some custom software is needed to really automate provisioning things. Whether as part of a custom ERPNext module, a custom WordPress plugin, or standalone code.

The problem is capacity to build such a system right now, although @kate has expressed interest in doing more programming, and it is somewhat motivating to me too (no promises right now!).

Potentially a lightweight system building on top of the existing pattern of “yaml in git repos” and ansible would be plausible. Perhaps using AWX (open source ansible tower providing a REST API, web UI, etc on of ansible) and GitLab CI pipelines. Maybe using dynamic ansible inventory somewhere along the way makes it possible… would need more exploring.

One advantage to that approach is it’s building on top of the existing approaches, and at some point automated provisioning would be needed to take the hosting offering to the next level, even if the client facing part is not clear yet.

Conclusion

Well, not really a conclusion, as there is no clear outcome. Using our research and thinking to stimulate further discussion seems the way forward.

Thanks for all this Nick

It’s a lot of work, and GOOD that you’ve recommended us to hold back a bit on ERPNext (a recommendation to go ahead with anything is always be taken more skeptically!)

My intital thoughts:

If the link between automated provisioning and the (hypothetical) ERP systm was removed this might make it easier to come up with an ERP choice which suits us ?(question) for that " big list of things to tackle".

We’ve previously discussd and agreed we should offer Stripe and GoCardless as i understand they can be incorporated in to any site. So this isn’t a reason for going with non-FOSS such as Xero. I know it’s popular … but …

Woocommerce (WP plugin) I have heard good reports about .

I’d be interested to ask what other ERP systems you evaluatd please?

Cheers for now,
Jonathan

See this thread for the ones we looked at, of all the ones we found ERPNext does still look like the best choice.

After reading this thread on the CoTech forum, I wondered if CiviCRM would be an option to provide some of the needs here, especially given @Graham might have some useful input.

It might only provide these two though:

  • central client list with contact information, address, etc
  • co-op membership management, shares, etc.

… I don’t know much about the drupal integration of CiviCRM, but maybe that’s a route to adding other drupal features (e.g. e-commerce).

Just a thought rather than a deep exploration.

Hi there. Thanks for flagging this up @nicksellen, I wasn’t aware of the discussion. Nice to see you, albeit briefly, in Sheffield at the recent do.

Looking very briefly at the requirements, I’d say that CiviCRM could handle a fair bit of this, though not all.
Given the relatively limited stuff that Webarch sells, I’d argue that a full-blown e-commerce set-up is overkill, or at least something that could be handled as a later phase. CiviCRM, plugged into either a WP, or preferably a Drupal CMS (richer integration options), could give you the ability to set out various products/services and handle online ordering with fairly rich pricing options, payment processing (Stripe, GoCardless), acknowledgement/notification emails, all key data entered into the CRM database for future marketing and customer service interactions. It can create invoices and handle VAT. It can integrate with book-keeping accounting systems: stuff like Quickbooks via batch data export and then import into your accounting tool. There is a live integration with Xero (I’m just looking at that for a client).
Recurring billing: a bit trickier, although the payment processors can certainly handle the payments side of it. Both CiviCRM and Drupal provide lots of trigger/condition/action tools so there’s decent scope for automation of things.
Great for bulk mailing.
Could potentially be used as a customer service ticketing/case management tool, but would need a bit of work to set that up elegantly.
Membership management - yes. Share register - possibly, although we know that Jon at sharenergy has written an extension that can do that.
Happy to have a more detailed conversation. Hope this is of interest. Would love to help Webarch on this sort of thing. Can save a lot of time once up and running.

Just a thought in terms of automating provisioning. If you could write a script that parsed an email to extract the key data and acted on that, this could be a quick and rather dirty option?