Ulf,
I recommend reading the document from Jason and us here at TAMU for our rancher/kubernetes deployment. While specific to rancher, it should work on any kubernetes deployment.
https://github.com/folio-org/folio-install/tree/kube-rancher/alternative-install/kubernetes-rancher/TAMU
See this section for okapi clustering:
https://github.com/folio-org/folio-install/tree/kube-rancher/alternative-install/kubernetes-rancher/TAMU#okapi-notes
We had the okapi developers add a plugin to okapi for hazelcast to use kubernetes/etcd for cluster discovery.
Brandon Tharp | Systems Administrator III
Digital Initiatives | Texas A&M University Libraries
Tel. 979.458.5760 | Fax 979.845.6238
On 3/13/19, 11:31 AM, "sysops-sig@ole-lists.openlibraryfoundation.org on behalf of Jason Root" <sysops-sig@ole-lists.openlibraryfoundation.org on behalf of jroot@library.tamu.edu> wrote:
Ulf, I will be out for the rest of the week, however I would love to talk with you more about this.
I believe you have essentially ran into the same issues we have with such a system. Issues we hope to have addressed by some further Okapi development and integration with Kubernetes and/or other orchestration toolset.
-Jason R.
Sent from my iPhone
> On Mar 13, 2019, at 11:25 AM, Ulf Seltmann <seltmann@ub.uni-leipzig.de> wrote:
>
> Hi all,
>
> While creating a workflow to deploy folio in a kubernetes-cluster i
> stumbled upon some principals of folio which i find rather complicated.
> Since it might also be a philosophical manner I want to discuss it here
> instead of the slack-chat (also it is a lot of text).
>
> # Okapi-Registry
> One thing is the "global registry"[1] which holds a description of all
> backend- and frontend-modules in all versions.
> As I understand it its purpose is to provide every okapi with a list of
> all installable modules (frontend and backen) in all versions, so okapi
> can automatically deploy the modules needed by a tenant into its own
> docker-daemon.
> In a single-server environment this might be a comfortable solution to
> a manual setup where an admin has to create each backend-module in
> particular.
> In a cluster-setup on the other hand i find it difficult to give okapi
> access to an underlying cluster or let okapi having its "own cluster"
> within the cluster.
> In my setup I create all backend-modules myself, and here i find it
> redundant to let okapi know of all installable the modules. Since i
> take care of all installed modules okapi does not need to know which
> modules exists afterall.
> My deployment-process for backend-modules right now looks like this:
> [2]. The problem is that i have to create a new docker-image for each
> backend-module in order to wrap the helper-script in it. As you can
> imagine i would like to omit it and just use the folio-images instead.
>
> # Okapi-Cluster
> The docs I found so far[3] explain running okapi in cluster-mode by
> using a cluster-toolkit called hazelcast. It describes clustering on a
> single machine and on separate machines.
> There are some issues i have with this cluster-solution:
> - I as admin have to decide on which okapi-node a backend-module is
> deployed
> - One Okapi-node is acting as proxy and is proxying all traffic to the
> responsible okapi-node (the one on which the backend-module is
> deployed), so there is no load-balancing possible between the okapi-
> nodes
> - Okapi itself seems to be stateful with its registered modules, so the
> application and cannot be load-balanced at startup. However i could
> loadbalance okapi by increasing its replicas after all modules were
> registered. Perhaps it is an easy task to make module-data storage-
> agnostic as well.
>
> In ideal an application should be infrastructure-agnostic and process
> what it gets with the information it has. If the state is part of that
> process it should be available to the application and each of its
> replicas.
>
> I would like to hear your opinion about it and please correct me if I
> am wrong. Also I intend to participate in the conference-call on friday
> .
>
> Regards
> Ulf
>
> [1]:
http://folio-registry.aws.indexdata.com
> [2]:
https://tinyurl.com/y6l3mxvu
> [3]:
>
https://github.com/folio-org/okapi/blob/5fa2880fccd1b3d9013e89fed401bd7233b8b994/doc/guide.md#running-in-cluster-mode
>
> --
> Ulf Seltmann
> Bereich Digitale Dienste
> Webmaster
>
> Universitätsbibliothek Leipzig
> Beethovenstraße 6, 04107 Leipzig
>
> T +49 341 97 30624
>
> seltmann@ub.uni-leipzig.de
> www.ub.uni-leipzig.de
>
> ------------------------------------------------------
> You received this message because you are subscribed to OLE Mailing List
> "sysops-sig".
> To unsubscribe from this list and stop receiving emails from it, follow
> this link:
http://archives.simplelists.com.
> To post to this group, send email to
> sysops-sig@ole-lists.openlibraryfoundation.org
> <mailto:sysops-sig@ole-lists.openlibraryfoundation.org>.
> Visit this group at
>
https://ole-lists.openlibraryfoundation.org<
https://ole-lists.openlibraryfoundation.org>
> .
------------------------------------------------------
You received this message because you are subscribed to OLE Mailing List
"sysops-sig".
To unsubscribe from this list and stop receiving emails from it, follow
this link:
http://archives.simplelists.com.
To post to this group, send email to
sysops-sig@ole-lists.openlibraryfoundation.org
<mailto:sysops-sig@ole-lists.openlibraryfoundation.org>.
Visit this group at
https://ole-lists.openlibraryfoundation.org<
https://ole-lists.openlibraryfoundation.org>
.
------------------------------------------------------
You received this message because you are subscribed to OLE Mailing List
"sysops-sig".
To unsubscribe from this list and stop receiving emails from it, follow
this link:
http://www.simplelists.com/confirm.php?u=KYIsz7p1siM0gUzhMUY9o9tp6etcrnId.
To post to this group, send email to
sysops-sig@ole-lists.openlibraryfoundation.org
<mailto:sysops-sig@ole-lists.openlibraryfoundation.org>.
Visit this group at
https://ole-lists.openlibraryfoundation.org<
https://ole-lists.openlibraryfoundation.org>
.