Skip to main content

Resource Allocation

A Kubernetes master node can act as a Forward worker node; the two are independent.

Any node not tagged with the fwd-master label can host worker pods.

  • One Node: All Forward Cluster components, including compute and search workers, run on the single node, regardless of configuration settings.
  • Two Nodes: The Forward worker node will run both compute and search workers, regardless of settings. The app server and collector run on the master node, providing some degree of performance isolation.
  • Three or more Nodes: You can choose between two modes of allocating resources to search and compute workers:
    • SHARED: Each worker node runs both compute and search worker pods, with available VM memory split roughly equally between the two pods. This mode enables multiple searches to run in parallel, as well as enables lower snapshot processing time.
    • ISOLATED: Search and compute worker pods run on separate worker nodes. The fraction of worker nodes allocated for compute versus search is configurable, as described in the modify allocation section below. The compute and search worker pods will each have access to nearly the entire VM memory on that node, thereby enabling larger snapshots to be processed and searched. In addition, performance effects from search work become independent from compute work, enabling more consistent snapshot processing and search result times.

View or Modify Resource Allocation​

Resource allocation can be viewed and modified using the Forward REST APIs described below.

note

Admin credentials are required to access endpoints below.

View Current Allocation​

Current configuration can be retrieved via GET /api/workers/operatorConfig, for example:

curl -k https://<appserver>/api/workers/operatorConfig -u <admin_user>:<admin_password>

This call returns the current configuration in JSON format, for example:

{"mode":"ISOLATED","computeNodesShare":0.99}

The configuration includes two properties:

  • mode: Either SHARED or ISOLATED. The default is SHARED.
  • computeNodesShare: Applicable only in ISOLATED mode. Specifies the fraction of worker nodes dedicated to compute workers. Valid values range from 0.01 and 0.99. The default is 0.5.

For example, if the cluster has 50 worker nodes and computeNodesShare is set to 0.99, then 49 nodes (that is, floor(0.99 × 50)) are allocated to compute workers and one node is allocated to search.

The number of search and compute workers cannot be less than one. For example, if the cluster has only one worker node and computeNodesShare is set to 0.99, both compute and search workers will run on the same node.

Modify Allocation​

You can update the configuration via PATCH /api/workers/operatorConfig, with the body of the config specified in JSON format, for example:

curl -k https://<appserver>/api/workers/operatorConfig \
  -u "<admin_user>:<admin_password>" \
  -X PATCH \
  -H "Content-Type: application/json" \
  --data '{"mode":"ISOLATED","computeNodesShare":0.5}'
warning

Before updating configuration settings, make sure the cluster is idle (that is, it is not collecting or processing snapshots). Any in-progress operations might fail.

Changes may take a few minutes to take effect. You can use the GET /api/workers/num endpoint to verify the number of workers.

Example:

curl -k https://<appserver>/api/workers/num