Initialization and Configuration

❗️

This is a legacy Apache Ignite documentation

The new documentation is hosted here: https://ignite.apache.org/docs/latest/

The below sections explain the basic steps to work with Apache Ignite using Python client.

Before connecting to Ignite from Python thin client, you must start at least one Ignite cluster node. For instance, you can use the ignite.sh script from ignite-version/bin as follows:

$ ./ignite.sh

$ ignite.bat

Connecting to the Cluster

The following code snippet shows how to connect to an Ignite cluster from Python thin client:

from pyignite import Client

## Open a connection
client = Client()
client.connect('127.0.0.1', 10800)

Creating a Cache

Using Python thin client, you can create a cache in the Ignite cluster, like so:

from pyignite import Client

## Open a connection
client = Client()
client.connect('127.0.0.1', 10800)

## Create a cache
my_cache = client.create_cache('my cache')

📘

Python example files

Python thin client contains fully workable examples to demonstrate the behavior of the client.

Configuring a Cache

The prop_codes module contains a list of ordinal values, that represent various cache settings.

Please refer to the Data Grid documentation on cache synchronization, rebalance, affinity and other cache configuration-related details.

The following read/write cache properties can be used to configure a cache via create_cache() or get_or_create_cache().

Example

cache_config = {
  PROP_NAME: 'my_cache',
  PROP_CACHE_KEY_CONFIGURATION: [
    {
      'type_name': 'my_type',
      'affinity_key_field_name': 'my_field',
    },
  ],
}
my_cache = client.create_cache(cache_config)

Query entity

• table_name: SQL table name.
• key_field_name: name of the key field.
• key_type_name: name of the key type (Java type or complex object).
• value_field_name: name of the value field.
• value_type_name: name of the value type.
• field_name_aliases: a list of 0 or more dicts of aliases (see Field name alias).
• query_fields: a list of 0 or more query field names (see Query field).
• query_indexes: a list of 0 or more query indexes (see Query index).

Field name alias

• field_name: field name.
• alias: alias (str).

Query field

• name: field name.
• type_name: name of Java type or complex object.
• is_key_field: (optional) boolean value, False by default.
• is_notnull_constraint_field: boolean value.
• default_value: (optional) anything that can be converted to type_name type. None (Null) by default.
• precision − (optional) decimal precision: total number of digits in decimal value. Defaults to -1 (use cluster default). Ignored for non-decimal SQL types (other than java.math.BigDecimal).
• scale − (optional) decimal precision: number of digits after the decimal point. Defaults to -1 (use cluster default). Ignored for non-decimal SQL types.

Query index

• index_name: index name.
• index_type: index type code as an integer value in unsigned byte range.
• inline_size: integer value.
• fields: a list of 0 or more indexed fields (see Fields).

Fields

• name: field name.
• is_descending: (optional) boolean value; False by default.

Cache key

• type_name: name of the complex object.
• affinity_key_field_name: name of the affinity key field.

Data Types

Apache Ignite uses a sophisticated system of serializable data types to store and retrieve user data, as well as to manage the configuration of its caches through the Ignite binary protocol.

Most of Ignite data types can be represented by the standard Python data type or class. Some of them, however, are conceptually alien, overly complex, or ambiguous to Python dynamic type system.

The following table summarizes the notion of Apache Ignite data types, as well as their representation and handling in Python. Note that parser/constructor classes are not instantiable. You are not obliged to use those parser/constructor classes. Pythonic types will suffice to interact with the Apache Ignite binary API. However, in some rare cases of type ambiguity, as well as for interoperability, you may have to sneak in one or the other class, along with your data, to some API function as a type conversion hint.

Primitive Data Types

Standard Objects

📘

Timestamp Precision

Python uses microsecond precision for timestamps, but Ignite uses nanosecond precision. This leads to a mismatch between expected results. When sending a timestamp to Ignite, be sure to return a date time and a three digit integer representing the delta in nanoseconds (4 byte integer, 0 - 999 range used). Example:

cache_put(client, cache, 'my_key', (datetime(year=1998, month=4, day=6, hour=18, minute=30), 500), value_hint=TimestampObject)

Arrays of Primitives

Arrays of standard objects

Object Collections, Special Types, and Complex Object

Failover

When connection to the server is broken or timed out, Client object propagates an original exception (OSError or SocketError), but keeps its constructor’s parameters intact and tries to reconnect transparently.

When there’s no way for the Client to reconnect, it raises a special ReconnectError exception.

The following example features a simple node list traversal failover mechanism. Gather 3 Ignite nodes on localhost into one cluster and run:

from pyignite import Client
from pyignite.datatypes.cache_config import CacheMode
from pyignite.datatypes.prop_codes import *
from pyignite.exceptions import SocketError


nodes = [
    ('127.0.0.1', 10800),
    ('127.0.0.1', 10801),
    ('127.0.0.1', 10802),
]

client = Client(timeout=4.0)
client.connect(nodes)
print('Connected to {}'.format(client))

my_cache = client.get_or_create_cache({
    PROP_NAME: 'my_cache',
    PROP_CACHE_MODE: CacheMode.REPLICATED,
})
my_cache.put('test_key', 0)

# abstract main loop
while True:
    try:
        # do the work
        test_value = my_cache.get('test_key')
        my_cache.put('test_key', test_value + 1)
    except (OSError, SocketError) as e:
        # recover from error (repeat last command, check data
        # consistency or just continue − depends on the task)
        print('Error: {}'.format(e))
        print('Last value: {}'.format(my_cache.get('test_key')))
        print('Reconnected to {}'.format(client))

Then try shutting down and restarting nodes, and see what happens.

# Connected to 127.0.0.1:10800
# Error: [Errno 104] Connection reset by peer
# Last value: 6999
# Reconnected to 127.0.0.1:10801
# Error: Socket connection broken.
# Last value: 12302
# Reconnected to 127.0.0.1:10802
# Error: [Errno 111] Client refused
# Traceback (most recent call last):
#     ...
# pyignite.exceptions.ReconnectError: Can not reconnect: out of nodes

Client reconnection does not require an explicit user action, like calling a special method or resetting a parameter. Note, however, that reconnection is lazy: it happens only if (and when) it is needed. In this example, the automatic reconnection happens when the script checks upon the last saved value:

print('Last value: {}'.format(my_cache.get('test_key')))

It means that instead of checking the connection status it is better for pyignite user to just try the supposed data operations and catch the resulting exception.

The connect() method accepts any iterable, not just list. It means that you can implement any reconnection policy (round-robin, nodes prioritization, pause on reconnect or graceful backoff) with a generator.

pyignite comes with a sample RoundRobin generator. In the above example try to replace

client.connect(nodes)

with

client.connect(RoundRobin(nodes, max_reconnects=20))

The client will try to reconnect to node 1 after node 3 is crashed, then to node 2, etc. At least one node should be active for the RoundRobin to work properly.