I was thinking that the analysis that I outlined below would include the HOW… perhaps too briefly put: "Once past that, we’d like to understand the best ways to integrate with and leverage Camunda.”

I don’t think the tech council could or should dive into enough detail to make a properly informed decision.

So the estimates we are trying to come up with represent how much time is needed to fully investigate Camunda, design its method(s) of integration with FOLIO and then the work to make that happen. Perhaps you are saying the design needs to be separated from the work to make that happen. If so, fair point, but I’d like to see if we can try and provide some sense of scope.

-mdg



On Aug 10, 2018, at 12:36 PM, Vince Bareau <vbareau@ebsco.com> wrote:

With regard to "the third issue" (Workflow), I think that the conversation needs to include a discussion of HOW Camunda might integrate with Folio and furthermore that this would be a prerequisite to any discussion of estimates. Since logically estimates would be different for different approaches.

To be more specific here's what I'm thinking:
  • based on everything I've seen and tried so far on Camunda, I like that selection (i.e. until proven otherwise, I'm inclined to believe that it is the right choice for Folio)
  • what concerns me is the sense of urgency expressed during the PC meeting yesterday. The impression was: we need to start this right away because app/modules need to integrate with Camunda/WFE sooner rather than later.
    • my concern is that this implies a tightly coupled integration between Folio and Camunda: i.e. modules need to integrate to Camunda.
    • this leads us down the same path as OLE's early and deep integration of a worklfow engine with RICE, which was a mistake.
    • there are other ways to integrate Camunda more loosely to Folio.
  • there needs to be a distinction made between using a workflow engine for data coordination between µservices and human task management.
    • data coordination between µservices, by example, is how an inventory stub item is created when an order is created in acquisitions
    • human task management, by example, is notifying a staff member than an approval is required in a workflow.
      • I'll go on a slight tangent here and point out that there is another risk of tight coupling should BPM workflows be created that are directly tied to individual users in various processes or tasks. BPM steps should remain ignorant of specific users and be abstracted to Roles (which are still missing in Folio). But that's another discussion...
    • the criticality of the two are different in that there are many possible workarounds for the second (e.g. email, Folio notifications, Trello, etc.. ) but not so much for the first one.
  • Camunda offers the possibility for a loosely coupled integration.
    • Camunda is essentially a standalone workflow engine that can be driven from a workflow definition in the form of a declaration: BPMN 2.0 diagram as an XML artifact. You create a BPM workflow as an artifact and feed it to the engine.
    • This is an ideal point for decoupling between the execution engine on the backend and the design/configuration tools used to define and manage workflows.
    • Camunda has a mature and very usable Modeler for creating the BPM artifact. I see this as taking a lot of the pressure off of the WFE discussion. Essentially the WFE mockups discussed so far come down to a homegrown, built-in Modeler for Folio whose purpose ultimately is to produce/modify a BPM workflow artifact that can be fed to Camunda. Therefore, the assertion to verify is that using Camunda Modeler is sufficient for meeting the workflow needs for v1. We can choose to recreate our own modeler in Folio later.
  • loose coupling of Camunda to Folio.
    • Rather than require all modules to be aware of, and integrate to, Camunda, it seems more desirable to me to have Camunda integrate with Folio without having modules and apps do anything.
      • this lowers the barrier for any new apps to be created in the system.
      • allows for integration to 3rd party systems that might otherwise not be able to comply
      • removes unnecessary urgency
      • does not lock in a particular workflow engine choice (e.e. Camunda).
    • To accomplish this sort of loose coupling:
      • Camunda integration is through a Folio module (e.g. mod-wfe-comunda)
      • mod-wfe-comunda monitors the transaction stream in Folio. It detects events that it recognizes as relevant to active workflows and triggers the workflow(s) accordingly in Camunda.
      • For example, the acquisition app creates a new order, which is detected by mod-wfe-comunda. It triggers a workflow that causes a stub record to inventory.  Neither the acquisition nor the inventory app is aware of Camunda.
      • Another example: a MARC record is uploaded into the BIB Store, mod-wfe-comunda detects it and triggers a workflow to send it to MARCcat for certification. When MARCcat is done, it puts it back into BIB Store, mod-wfe-comunda detects that and triggers the workflow to create an Inventory record from it.
      • Note that this would benefit from a push model for Okapi (i.e. AES)

_
V

On 2018-08-10 07:33, Mike Gorrell wrote:
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.


Folks,

Next week’s Tech Council (15-Aug, 11am Eastern) meeting will be full of important/timely issues, so I wanted to send an email and have us make as much progress/pre-meeting prep as possible. Additionally, I am traveling on Wednesday and my flight lands at 11:08am - I’ll join via mobile as soon as I can. Tod Olson has agreed to run the meeting in my absence.

The agenda is here: https://wiki.folio.org/display/TC/20180815+-+Aug+15%2C+2018+-+Technical+Council+Meeting+Notes

The first issue listed - Hardcoded lists - was raised at the Product Council yesterday. These two threads outline the potential issue of lists not being editable/customizable by libraries, which doesn’t seem like the way things were designed. Jakub has reached out to the teams to investigate and report back to the TC in case there are further actions that we need to take.
- https://discuss.folio.org/t/orders-material-types/1975
- https://discuss.folio.org/t/acquisitions-hardcoded-list-values/1937

The second issue is the AES Doc that Vince submitted before going on vacation: https://docs.google.com/document/d/15gWIh_iNZQvk0LzJwapofwnnyck1NM7h6BUTvWmCIDs/edit?usp=sharing
There seems to be good traction in the comments of the document and general consensus. I am hoping that by the time the meeting on Wednesday comes people are in 100% agreement.

The third issue is the one I think we’ll spend the most time on. This presentation was used to facilitate a conversation on Workflow in yesterday’s PC meeting:
https://docs.google.com/presentation/d/1vEZELmIlSC5AOusObnkMTyB0XkhWt_qNVcvLe0KilgI/edit?usp=sharing

After much discussion, the PC Asked us to validate/change the estimate I show on slide 11; how much work would it take to
1) Validate that Camunda is the right choice for use in FOLIO
2) Install Camunda and establish proper support so that the project can begin using it

When I put slide 11 together, this is what I was thinking:
A) We’d want a PO who could review the epics and features that have been identified as needing workflow, and compare those requirements to the capabilities of Camunda. This person would have to understand library workflows and the technical implementation of Camunda - likely installing and running the modeler and engine to fully validate it. They would identify gaps, and likely surplus capabilities that might feedback into what POs and librarians might want to leverage. This would all be documented. They would work with the other resources listed below to plan PoC work and generally structure/prioritize the effort.
B) We’d want a developer to understand the technical details of Camunda from its multi-tenant approach, how it fits with FOLIO’s backend architecture, etc, and identify any gotchas or troublesome ‘incompatibility’. Once past that, we’d like to understand the best ways to integrate with and leverage Camunda. Which APIs, settings, patterns, etc. In working through this I’d imagine there may be an opportunity and/or need to create some artifacts that can be used by teams who will eventually work with Camunda - a mini FOLIO-specific SDK of sorts?
C) We’d need a DevOps person for the backend plumbing setup, monitoring, and operation. There are also NFRs that need to be explored (performance, fault tolerance, upgrade process, etc.)
D) These resources would collaborate to create documentation that would permit a dev team to use Camunda as they would use STRIPES or Okapi

So the task of the TC is to review my logic and the roles and estimates that I put on slide 11.

I’ve invited Ian Ibbotson to join the meeting and to lead this discussion. He has the most experience with Camunda, having done the PoC, and he lead the dev team who is doing ERM - an application that has workflow needs.

SECONDARILY, if we could also discuss and amend the four estimates that were made on page 12, that would be great.
1) Once Camunda is up and in use throughout the community, how many resources are needed to keep it up and running? This is analogous to how many resources are needed to keep Stripes/Okapi healthy and supported for the community.
2) Assuming Camunda is ready - how long does it take a dev team to climb the curve to begin imbedding it in their app/modules?
3) Once an app/module is integrated with Camunda, it needs workflows to be authored. Each implementer of FOLIO needs to have staff that can author, run and monitor their workflows. How long and how many resources are reasonable for this training/learning curve.
4) On an operational basis, how much time does a library need to allocate to the care and feeding of their workflows. In my view this is incremental (but analogous) to the support a library would allocate to overall FOLIO. Just acknowledging that adding Camunda adds to the level of support required.

There’s one last item on the agenda - Consortia Model. I’ll spend some time early next week prepping for a discussion on that if we have time.

I’d like to ask you to think about these issues and ask questions/have discussions prior to our meeting Wednesday so that we can get through it all.

Thanks

-mdg




To unsubscribe from this list please go to http://archives.simplelists.com

To unsubscribe from this list please go to http://www.simplelists.com/confirm.php?u=SeK0ArgpLZB2ijVHc1q8eivZ4CXy8J2w