fdo-mirrors/NetworkManager
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/NetworkManager/NetworkManager.git synced 2025-12-25 23:30:07 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
NetworkManager/shared/nm-utils/nm-dedup-multi.h

438 lines
17 KiB
C
Raw Normal View History

shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
/* -*- Mode: C; tab-width: 4; indent-tabs-mode: t; c-basic-offset: 4 -*- */
/* NetworkManager -- Network link manager
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the
* Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
* Boston, MA 02110-1301 USA.
*
* (C) Copyright 2017 Red Hat, Inc.
*/
#ifndef __NM_DEDUP_MULTI_H__
#define __NM_DEDUP_MULTI_H__
#include "nm-obj.h"
core: track addresses for NMIP4Config/NMIP6Config via NMDedupMultiIndex Reasons: - it adds an O(1) lookup index for accessing NMIPxConfig's addresses. Hence, operations like merge/intersect have now runtime O(n) instead of O(n^2). Arguably, we expect low numbers of addresses in general. For low numbers, the O(n^2) doesn't matter and quite likely in those cases the previous implementation was just fine -- maybe even faster. But the simple case works fine either way. It's important to scale well in the exceptional case. - the tracked objects can be shared between the various NMPI4Config, NMIP6Config instances with NMPlatform and everybody else. - the NMPObject can be treated generically, meaning it enables code to handle both IPv4 and IPv6, or addresses and routes. See for example _nm_ip_config_add_obj(). - I want core to evolve to somewhere where we don't keep copies of NMPlatformIP4Address, et al. instances. Instead they shall all be shared. I hope this will reduce memory consumption (although tracking a reference consumes some memory too). Also, it shortcuts nmp_object_equal() when comparing the same object. Calling nmp_object_equal() on the identical objects would be a common case after the hash function pre-evaluates equality.
2017-07-07 23:34:41 +02:00
#include "c-list-util.h"
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
/*****************************************************************************/
all: extend hash functions with an NMHashState argument We often want to cascade hashing, meaning, to combine the outcome of various hash functions in a larger hash. Instead of having each hash function return a guint hash value, accept a hash state argument. This saves the overhead of initializing and completing the intermediate hash states. It also avoids loosing entropy when we reduce the larger hash state into the intermediate guint hash value.
2017-10-17 11:53:08 +02:00
struct _NMHashState;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
typedef struct _NMDedupMultiObj NMDedupMultiObj;
typedef struct _NMDedupMultiObjClass NMDedupMultiObjClass;
typedef struct _NMDedupMultiIdxType NMDedupMultiIdxType;
typedef struct _NMDedupMultiIdxTypeClass NMDedupMultiIdxTypeClass;
typedef struct _NMDedupMultiEntry NMDedupMultiEntry;
typedef struct _NMDedupMultiHeadEntry NMDedupMultiHeadEntry;
typedef struct _NMDedupMultiIndex NMDedupMultiIndex;
typedef enum _NMDedupMultiIdxMode {
NM_DEDUP_MULTI_IDX_MODE_PREPEND,
NM_DEDUP_MULTI_IDX_MODE_PREPEND_FORCE,
/* append new objects to the end of the list.
* If the object is already in the cache, don't move it. */
NM_DEDUP_MULTI_IDX_MODE_APPEND,
/* like NM_DEDUP_MULTI_IDX_MODE_APPEND, but if the object
* is already in teh cache, move it to the end. */
NM_DEDUP_MULTI_IDX_MODE_APPEND_FORCE,
} NMDedupMultiIdxMode;
/*****************************************************************************/
struct _NMDedupMultiObj {
union {
NMObjBaseInst parent;
const NMDedupMultiObjClass *klass;
};
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 23:46:06 +02:00
NMDedupMultiIndex *_multi_idx;
guint _ref_count;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
};
struct _NMDedupMultiObjClass {
NMObjBaseClass parent;
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 23:46:06 +02:00
const NMDedupMultiObj *(*obj_clone) (const NMDedupMultiObj *obj);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 23:46:06 +02:00
gboolean (*obj_needs_clone) (const NMDedupMultiObj *obj);
void (*obj_destroy) (NMDedupMultiObj *obj);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
all: extend hash functions with an NMHashState argument We often want to cascade hashing, meaning, to combine the outcome of various hash functions in a larger hash. Instead of having each hash function return a guint hash value, accept a hash state argument. This saves the overhead of initializing and completing the intermediate hash states. It also avoids loosing entropy when we reduce the larger hash state into the intermediate guint hash value.
2017-10-17 11:53:08 +02:00
/* the NMDedupMultiObj can be deduplicated. For that the obj_full_hash_update()
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
* and obj_full_equal() compare *all* fields of the object, even minor ones. */
all: extend hash functions with an NMHashState argument We often want to cascade hashing, meaning, to combine the outcome of various hash functions in a larger hash. Instead of having each hash function return a guint hash value, accept a hash state argument. This saves the overhead of initializing and completing the intermediate hash states. It also avoids loosing entropy when we reduce the larger hash state into the intermediate guint hash value.
2017-10-17 11:53:08 +02:00
void (*obj_full_hash_update) (const NMDedupMultiObj *obj,
struct _NMHashState *h);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
gboolean (*obj_full_equal) (const NMDedupMultiObj *obj_a,
const NMDedupMultiObj *obj_b);
};
/*****************************************************************************/
shared: move nm_dedup_multi_obj_ref() to header as inline nm_dedup_multi_obj_ref() is a trivial function, that only uses the field which is already declared in the same header file. Move it to the header so that it can be inlined (without LTO).
2017-09-22 13:38:41 +02:00
static inline const NMDedupMultiObj *
nm_dedup_multi_obj_ref (const NMDedupMultiObj *obj)
{
/* ref and unref accept const pointers. Objects is supposed to be shared
* and kept immutable. Disallowing to take/retrun a reference to a const
* NMPObject is cumbersome, because callers are precisely expected to
* keep a ref on the otherwise immutable object. */
nm_assert (obj);
nm_assert (obj->_ref_count != NM_OBJ_REF_COUNT_STACKINIT);
nm_assert (obj->_ref_count > 0);
((NMDedupMultiObj *) obj)->_ref_count++;
return obj;
}
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 23:46:06 +02:00
const NMDedupMultiObj *nm_dedup_multi_obj_unref (const NMDedupMultiObj *obj);
const NMDedupMultiObj *nm_dedup_multi_obj_clone (const NMDedupMultiObj *obj);
gboolean nm_dedup_multi_obj_needs_clone (const NMDedupMultiObj *obj);
gconstpointer nm_dedup_multi_index_obj_intern (NMDedupMultiIndex *self,
/* const NMDedupMultiObj * */ gconstpointer obj);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 23:46:06 +02:00
void nm_dedup_multi_index_obj_release (NMDedupMultiIndex *self,
/* const NMDedupMultiObj * */ gconstpointer obj);
/* const NMDedupMultiObj * */ gconstpointer nm_dedup_multi_index_obj_find (NMDedupMultiIndex *self,
/* const NMDedupMultiObj * */ gconstpointer obj);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
/*****************************************************************************/
/* the NMDedupMultiIdxType is an access handle under which you can store and
* retrieve NMDedupMultiObj instances in NMDedupMultiIndex.
*
* The NMDedupMultiIdxTypeClass determines it's behavior, but you can have
* multiple instances (of the same class).
*
* For example, NMIP4Config can have idx-type to put there all IPv4 Routes.
* This idx-type instance is private to the NMIP4Config instance. Basically,
* the NMIP4Config instance uses the idx-type to maintain an ordered list
* of routes in NMDedupMultiIndex.
*
* However, a NMDedupMultiIdxType may also partition the set of objects
* in multiple distinct lists. NMIP4Config doesn't do that (because instead
* of creating one idx-type for IPv4 and IPv6 routes, it just cretaes
* to distinct idx-types, one for each address family.
* This partitioning is used by NMPlatform to maintain a lookup index for
* routes by ifindex. As the ifindex is dynamic, it does not create an
* idx-type instance for each ifindex. Instead, it has one idx-type for
* all routes. But whenever accessing NMDedupMultiIndex with an NMDedupMultiObj,
* the partitioning NMDedupMultiIdxType takes into accound the NMDedupMultiObj
* instance to associate it with the right list.
*
* Hence, a NMDedupMultiIdxEntry has a list of possibly multiple NMDedupMultiHeadEntry
* instances, which each is the head for a list of NMDedupMultiEntry instances.
* In the platform example, the NMDedupMultiHeadEntry parition the indexed objects
* by their ifindex. */
struct _NMDedupMultiIdxType {
union {
NMObjBaseInst parent;
const NMDedupMultiIdxTypeClass *klass;
};
CList lst_idx_head;
guint len;
};
void nm_dedup_multi_idx_type_init (NMDedupMultiIdxType *idx_type,
const NMDedupMultiIdxTypeClass *klass);
struct _NMDedupMultiIdxTypeClass {
NMObjBaseClass parent;
all: extend hash functions with an NMHashState argument We often want to cascade hashing, meaning, to combine the outcome of various hash functions in a larger hash. Instead of having each hash function return a guint hash value, accept a hash state argument. This saves the overhead of initializing and completing the intermediate hash states. It also avoids loosing entropy when we reduce the larger hash state into the intermediate guint hash value.
2017-10-17 11:53:08 +02:00
void (*idx_obj_id_hash_update) (const NMDedupMultiIdxType *idx_type,
const NMDedupMultiObj *obj,
struct _NMHashState *h);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
gboolean (*idx_obj_id_equal) (const NMDedupMultiIdxType *idx_type,
const NMDedupMultiObj *obj_a,
const NMDedupMultiObj *obj_b);
/* an NMDedupMultiIdxTypeClass which implements partitioning of the
* tracked objects, must implement the idx_obj_partition*() functions.
*
* idx_obj_partitionable() may return NULL if the object cannot be tracked.
* For example, a index for routes by ifindex, may not want to track any
* routes that don't have a valid ifindex. If the idx-type says that the
* object is not partitionable, it is never added to the NMDedupMultiIndex. */
gboolean (*idx_obj_partitionable) (const NMDedupMultiIdxType *idx_type,
const NMDedupMultiObj *obj);
all: extend hash functions with an NMHashState argument We often want to cascade hashing, meaning, to combine the outcome of various hash functions in a larger hash. Instead of having each hash function return a guint hash value, accept a hash state argument. This saves the overhead of initializing and completing the intermediate hash states. It also avoids loosing entropy when we reduce the larger hash state into the intermediate guint hash value.
2017-10-17 11:53:08 +02:00
void (*idx_obj_partition_hash_update) (const NMDedupMultiIdxType *idx_type,
const NMDedupMultiObj *obj,
struct _NMHashState *h);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
gboolean (*idx_obj_partition_equal) (const NMDedupMultiIdxType *idx_type,
const NMDedupMultiObj *obj_a,
const NMDedupMultiObj *obj_b);
};
static inline gboolean
nm_dedup_multi_idx_type_id_equal (const NMDedupMultiIdxType *idx_type,
/* const NMDedupMultiObj * */ gconstpointer obj_a,
/* const NMDedupMultiObj * */ gconstpointer obj_b)
{
nm_assert (idx_type);
core: shortcut equal operator for identical object reference in NMDedupMultiIndex And get rid of the unused obj_full_equality_allows_different_class. It's hard to grasp how to implement different object types that can compare despite having different klasses. The idea was, that stack allocated objects (used as lookup needles), are some small lightweight objects, that still compare equal to the full instance. But it's unused. Drop it.
2017-07-10 16:36:58 +02:00
return obj_a == obj_b
|| idx_type->klass->idx_obj_id_equal (idx_type,
obj_a,
obj_b);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
}
static inline gboolean
nm_dedup_multi_idx_type_partition_equal (const NMDedupMultiIdxType *idx_type,
/* const NMDedupMultiObj * */ gconstpointer obj_a,
/* const NMDedupMultiObj * */ gconstpointer obj_b)
{
nm_assert (idx_type);
if (idx_type->klass->idx_obj_partition_equal) {
nm_assert (obj_a);
nm_assert (obj_b);
core: shortcut equal operator for identical object reference in NMDedupMultiIndex And get rid of the unused obj_full_equality_allows_different_class. It's hard to grasp how to implement different object types that can compare despite having different klasses. The idea was, that stack allocated objects (used as lookup needles), are some small lightweight objects, that still compare equal to the full instance. But it's unused. Drop it.
2017-07-10 16:36:58 +02:00
return obj_a == obj_b
|| idx_type->klass->idx_obj_partition_equal (idx_type,
obj_a,
obj_b);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
}
return TRUE;
}
/*****************************************************************************/
struct _NMDedupMultiEntry {
/* this is the list of all entries that share the same head entry.
* All entries compare equal according to idx_obj_partition_equal(). */
CList lst_entries;
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 23:46:06 +02:00
/* const NMDedupMultiObj * */ gconstpointer obj;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
bool is_head;
bool dirty;
const NMDedupMultiHeadEntry *head;
};
struct _NMDedupMultiHeadEntry {
/* this is the list of all entries that share the same head entry.
* All entries compare equal according to idx_obj_partition_equal(). */
CList lst_entries_head;
const NMDedupMultiIdxType *idx_type;
bool is_head;
guint len;
CList lst_idx;
};
platform: cleanup lookup API for objects in NMPCache
2017-08-05 09:57:30 +02:00
/*****************************************************************************/
static inline gconstpointer
nm_dedup_multi_entry_get_obj (const NMDedupMultiEntry *entry)
{
/* convenience method that allows to skip the %NULL check on
* @entry. Think of the NULL-conditional operator ?. of C# */
return entry ? entry->obj : NULL;
}
/*****************************************************************************/
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
static inline void
nm_dedup_multi_entry_set_dirty (const NMDedupMultiEntry *entry,
gboolean dirty)
{
/* NMDedupMultiEntry is always exposed as a const object, because it is not
* supposed to be modified outside NMDedupMultiIndex API. Except the "dirty"
* flag. In C++ speak, it is a mutable field.
*
core: fix clearing dirty flag in _nm_ip_config_add_obj() We rely on clearing the dirty flag. For example in nm_ip4_config_replace(), we first mark all entries dirty, then force-append the ones we keep, and finally remove the ones that are still dirty. Since _nm_ip_config_add_obj() short-cuts nm_dedup_multi_index_add_full(), it must clear the dirty flag on its own.
2017-08-31 12:51:48 +02:00
* Add this inline function, to cast-away constness and set the dirty flag. */
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
nm_assert (entry);
((NMDedupMultiEntry *) entry)->dirty = dirty;
}
/*****************************************************************************/
NMDedupMultiIndex *nm_dedup_multi_index_new (void);
NMDedupMultiIndex *nm_dedup_multi_index_ref (NMDedupMultiIndex *self);
NMDedupMultiIndex *nm_dedup_multi_index_unref (NMDedupMultiIndex *self);
static inline void
_nm_auto_unref_dedup_multi_index (NMDedupMultiIndex **v)
{
if (*v)
nm_dedup_multi_index_unref (*v);
}
#define nm_auto_unref_dedup_multi_index nm_auto(_nm_auto_unref_dedup_multi_index)
#define NM_DEDUP_MULTI_ENTRY_MISSING ((const NMDedupMultiEntry *) GUINT_TO_POINTER (1))
#define NM_DEDUP_MULTI_HEAD_ENTRY_MISSING ((const NMDedupMultiHeadEntry *) GUINT_TO_POINTER (1))
gboolean nm_dedup_multi_index_add_full (NMDedupMultiIndex *self,
NMDedupMultiIdxType *idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj,
NMDedupMultiIdxMode mode,
const NMDedupMultiEntry *entry_order,
const NMDedupMultiEntry *entry_existing,
const NMDedupMultiHeadEntry *head_existing,
const NMDedupMultiEntry **out_entry,
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 23:46:06 +02:00
/* const NMDedupMultiObj ** */ gpointer out_obj_old);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
gboolean nm_dedup_multi_index_add (NMDedupMultiIndex *self,
NMDedupMultiIdxType *idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj,
NMDedupMultiIdxMode mode,
const NMDedupMultiEntry **out_entry,
core: remove NMDedupMultiBox object and track NMDedupMultiObj instances directly Implement the reference counting of NMPObject as part of NMDedupMultiObj and get rid of NMDedupMultiBox. With this change, the NMPObject is aware in which NMDedupMultiIndex instance it is tracked. - this saves an additional GSlice allocation for the NMDedupMultiBox. - it is immediately known, whether an NMPObject is tracked by a certain NMDedupMultiIndex or not. This saves an additional hash lookup. - previously, when all idx-types cease to reference an NMDedupMultiObj instance, it was removed. Now, a tracked objects stays in the NMDedupMultiIndex until it's last reference is deleted. This possibly extends the lifetime of the object and we may reuse it better. - it is no longer possible to add one object to more then one NMDedupMultiIndex instance. As we anyway want to have only one instance to deduplicate the objects, this is fine. - the ref-counting implementation is now part of NMDedupMultiObj. Previously, NMDedupMultiIndex could also track objects that were not ref-counted. Hoever, the object anyway *must* implement the NMDedupMultiObj API, so this flexibility is unneeded and was not used. - a downside is, that NMPObject grows by one pointer size, even if it isn't tracked in the NMDedupMultiIndex. But we really want to put all objects into the index for sharing and deduplication. So this downside should be acceptable. Still, code like nmp_object_stackinit*() needs to handle a larger object.
2017-07-02 23:46:06 +02:00
/* const NMDedupMultiObj ** */ gpointer out_obj_old);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
platform: add non-exclusive routes and drop route-manager Previously, we would add exclusive routes via netlink message flags NLM_F_CREATE | NLM_F_REPLACE for RTM_NEWROUTE. Similar to `ip route replace`. Using that form of RTM_NEWROUTE message, we could only add a certain route with a certain network/plen,metric triple once. That was already hugely inconvenient, because - when configuring routes, multiple (managed) interfaces may get conflicting routes (multihoming). Only one of the routes can be actually configured using `ip route replace`, so we need to track routes that are currently shadowed. - when configuring routes, we might replace externally configured routes on unmanaged interfaces. We should not interfere with such routes. That was worked around by having NMRouteManager (and NMDefaultRouteManager). NMRouteManager would keep a list of the routes which NetworkManager would like to configure, even if momentarily being unable to do so due to conflicting routes. This worked mostly well but was complicated. It involved bumping metrics to avoid conflicts for device routes, as we might require them for gateway routes. Drop that now. Instead, use the corresponding of `ip route append` to configure routes. This allows NetworkManager to confiure (almost) all routes that we care. Especially, it can configure all routes on a managed interface, without replacing/interfering with routes on other interfaces. Hence, NMRouteManager becomes obsolete. It practice it is a bit more complicated because: - when adding an IPv4 address, kernel will automatically create a device route for the subnet. We should avoid that by using the IFA_F_NOPREFIXROUTE flag for IPv4 addresses (still to-do). But as kernel may not support that flag for IPv4 addresses yet (and we don't require such a kernel yet), we still need functionality similar to nm_route_manager_ip4_route_register_device_route_purge_list(). This functionality is now handled via nm_platform_ip4_dev_route_blacklist_set(). - trying to configure an IPv6 route with a source address will be rejected by kernel as long as the address is tentative (see related bug rh#1457196). Preferably, NMDevice would keep the list of routes which should be configured, while kernel would have the list of what actually is configured. There is a feed-back loop where both affect each other (for example, when externally deleting a route, NMDevice must forget about it too). Previously, NMRouteManager would have the task of remembering all routes which we currently want to configure, but cannot due to conflicting routes. We get rid of that, because now we configure non-exclusive routes. We however still will need to remember IPv6 routes with a source address, that currently cannot be configured yet. Hence, we will need to keep track of routes that currently cannot be configured, but later may be. That is still not done yet, as NMRouteManager didn't handle this correctly either.
2017-08-14 14:18:53 +02:00
const NMDedupMultiEntry *nm_dedup_multi_index_lookup_obj (const NMDedupMultiIndex *self,
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
const NMDedupMultiIdxType *idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj);
platform: add non-exclusive routes and drop route-manager Previously, we would add exclusive routes via netlink message flags NLM_F_CREATE | NLM_F_REPLACE for RTM_NEWROUTE. Similar to `ip route replace`. Using that form of RTM_NEWROUTE message, we could only add a certain route with a certain network/plen,metric triple once. That was already hugely inconvenient, because - when configuring routes, multiple (managed) interfaces may get conflicting routes (multihoming). Only one of the routes can be actually configured using `ip route replace`, so we need to track routes that are currently shadowed. - when configuring routes, we might replace externally configured routes on unmanaged interfaces. We should not interfere with such routes. That was worked around by having NMRouteManager (and NMDefaultRouteManager). NMRouteManager would keep a list of the routes which NetworkManager would like to configure, even if momentarily being unable to do so due to conflicting routes. This worked mostly well but was complicated. It involved bumping metrics to avoid conflicts for device routes, as we might require them for gateway routes. Drop that now. Instead, use the corresponding of `ip route append` to configure routes. This allows NetworkManager to confiure (almost) all routes that we care. Especially, it can configure all routes on a managed interface, without replacing/interfering with routes on other interfaces. Hence, NMRouteManager becomes obsolete. It practice it is a bit more complicated because: - when adding an IPv4 address, kernel will automatically create a device route for the subnet. We should avoid that by using the IFA_F_NOPREFIXROUTE flag for IPv4 addresses (still to-do). But as kernel may not support that flag for IPv4 addresses yet (and we don't require such a kernel yet), we still need functionality similar to nm_route_manager_ip4_route_register_device_route_purge_list(). This functionality is now handled via nm_platform_ip4_dev_route_blacklist_set(). - trying to configure an IPv6 route with a source address will be rejected by kernel as long as the address is tentative (see related bug rh#1457196). Preferably, NMDevice would keep the list of routes which should be configured, while kernel would have the list of what actually is configured. There is a feed-back loop where both affect each other (for example, when externally deleting a route, NMDevice must forget about it too). Previously, NMRouteManager would have the task of remembering all routes which we currently want to configure, but cannot due to conflicting routes. We get rid of that, because now we configure non-exclusive routes. We however still will need to remember IPv6 routes with a source address, that currently cannot be configured yet. Hence, we will need to keep track of routes that currently cannot be configured, but later may be. That is still not done yet, as NMRouteManager didn't handle this correctly either.
2017-08-14 14:18:53 +02:00
const NMDedupMultiHeadEntry *nm_dedup_multi_index_lookup_head (const NMDedupMultiIndex *self,
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
const NMDedupMultiIdxType *idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj);
guint nm_dedup_multi_index_remove_entry (NMDedupMultiIndex *self,
gconstpointer entry);
guint nm_dedup_multi_index_remove_obj (NMDedupMultiIndex *self,
NMDedupMultiIdxType *idx_type,
shared: return deleted object from nm_dedup_multi_index_remove_obj() For completeness of the API. remove_obj() is basically a shortcut of nm_dedup_multi_index_lookup_obj() combined with nm_dedup_multi_index_remove_entry(). As such, it is useful to return the actually deleted object. Note that the lookup needle @obj is not necessarily the same instance as the one that will be removed, it's only an instance that compares equal according to the index's equality operator.
2017-08-23 16:00:11 +02:00
/*const NMDedupMultiObj * */ gconstpointer obj,
/*const NMDedupMultiObj ** */ gconstpointer *out_obj);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
guint nm_dedup_multi_index_remove_head (NMDedupMultiIndex *self,
NMDedupMultiIdxType *idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj);
guint nm_dedup_multi_index_remove_idx (NMDedupMultiIndex *self,
NMDedupMultiIdxType *idx_type);
void nm_dedup_multi_index_dirty_set_head (NMDedupMultiIndex *self,
const NMDedupMultiIdxType *idx_type,
/*const NMDedupMultiObj * */ gconstpointer obj);
void nm_dedup_multi_index_dirty_set_idx (NMDedupMultiIndex *self,
const NMDedupMultiIdxType *idx_type);
guint nm_dedup_multi_index_dirty_remove_idx (NMDedupMultiIndex *self,
NMDedupMultiIdxType *idx_type,
gboolean mark_survivors_dirty);
/*****************************************************************************/
typedef struct _NMDedupMultiIter {
core: simplify NMDedupMultiIter by storing CList pointer Let next and head pointers point to the CList value, instead of NMDedupMultiEntry.
2017-07-16 19:18:51 +02:00
const CList *_head;
const CList *_next;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
const NMDedupMultiEntry *current;
} NMDedupMultiIter;
static inline void
nm_dedup_multi_iter_init (NMDedupMultiIter *iter, const NMDedupMultiHeadEntry *head)
{
g_return_if_fail (iter);
core: simplify NMDedupMultiIter by storing CList pointer Let next and head pointers point to the CList value, instead of NMDedupMultiEntry.
2017-07-16 19:18:51 +02:00
if (head && !c_list_is_empty (&head->lst_entries_head)) {
iter->_head = &head->lst_entries_head;
iter->_next = head->lst_entries_head.next;
} else {
iter->_head = NULL;
iter->_next = NULL;
}
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
iter->current = NULL;
}
static inline gboolean
nm_dedup_multi_iter_next (NMDedupMultiIter *iter)
{
g_return_val_if_fail (iter, FALSE);
core: simplify NMDedupMultiIter by storing CList pointer Let next and head pointers point to the CList value, instead of NMDedupMultiEntry.
2017-07-16 19:18:51 +02:00
if (!iter->_next)
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
return FALSE;
core: simplify NMDedupMultiIter by storing CList pointer Let next and head pointers point to the CList value, instead of NMDedupMultiEntry.
2017-07-16 19:18:51 +02:00
/* we always look ahead for the next. This way, the user
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
* may delete the current entry (but no other entries). */
core: simplify NMDedupMultiIter by storing CList pointer Let next and head pointers point to the CList value, instead of NMDedupMultiEntry.
2017-07-16 19:18:51 +02:00
iter->current = c_list_entry (iter->_next, NMDedupMultiEntry, lst_entries);
if (iter->_next->next == iter->_head)
iter->_next = NULL;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
else
core: simplify NMDedupMultiIter by storing CList pointer Let next and head pointers point to the CList value, instead of NMDedupMultiEntry.
2017-07-16 19:18:51 +02:00
iter->_next = iter->_next->next;
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
return TRUE;
}
platform: refactor nm_dedup_multi_objs_to_ptr_array_head() by moving the core functionality to "nm-dedup-multi.c". As the ref-counting mechanism now is part of "nm-dedup-multi.c", this works better and is reusable outside of platform.
2017-07-04 09:57:06 +02:00
#define nm_dedup_multi_iter_for_each(iter, head_entry) \
for (nm_dedup_multi_iter_init ((iter), (head_entry)); \
nm_dedup_multi_iter_next ((iter)); \
)
/*****************************************************************************/
typedef gboolean (*NMDedupMultiFcnSelectPredicate) (/* const NMDedupMultiObj * */ gconstpointer obj,
gpointer user_data);
core: track addresses for NMIP4Config/NMIP6Config via NMDedupMultiIndex Reasons: - it adds an O(1) lookup index for accessing NMIPxConfig's addresses. Hence, operations like merge/intersect have now runtime O(n) instead of O(n^2). Arguably, we expect low numbers of addresses in general. For low numbers, the O(n^2) doesn't matter and quite likely in those cases the previous implementation was just fine -- maybe even faster. But the simple case works fine either way. It's important to scale well in the exceptional case. - the tracked objects can be shared between the various NMPI4Config, NMIP6Config instances with NMPlatform and everybody else. - the NMPObject can be treated generically, meaning it enables code to handle both IPv4 and IPv6, or addresses and routes. See for example _nm_ip_config_add_obj(). - I want core to evolve to somewhere where we don't keep copies of NMPlatformIP4Address, et al. instances. Instead they shall all be shared. I hope this will reduce memory consumption (although tracking a reference consumes some memory too). Also, it shortcuts nmp_object_equal() when comparing the same object. Calling nmp_object_equal() on the identical objects would be a common case after the hash function pre-evaluates equality.
2017-07-07 23:34:41 +02:00
gconstpointer *nm_dedup_multi_objs_to_array_head (const NMDedupMultiHeadEntry *head_entry,
NMDedupMultiFcnSelectPredicate predicate,
gpointer user_data,
guint *out_len);
platform: refactor nm_dedup_multi_objs_to_ptr_array_head() by moving the core functionality to "nm-dedup-multi.c". As the ref-counting mechanism now is part of "nm-dedup-multi.c", this works better and is reusable outside of platform.
2017-07-04 09:57:06 +02:00
GPtrArray *nm_dedup_multi_objs_to_ptr_array_head (const NMDedupMultiHeadEntry *head_entry,
NMDedupMultiFcnSelectPredicate predicate,
gpointer user_data);
platform: fix cache to use kernel's notion for equality of routes Until now, NetworkManager's platform cache for routes used the quadruple network/plen,metric,ifindex for equaliy. That is not kernel's understanding of how routes behave. For example, with `ip route append` you can add two IPv4 routes that only differ by their gateway. To the previous form of platform cache, these two routes would wrongly look identical, as the cache could not contain both routes. This also easily leads to cache-inconsistencies. Now that we have NM_PLATFORM_IP_ROUTE_CMP_TYPE_ID, fix the route's compare operator to match kernel's. Well, not entirely. Kernel understands more properties for routes then NetworkManager. Some of these properties may also be part of the ID according to kernel. To NetworkManager such routes would still look identical as they only differ in a property that is not understood. This can still cause cache-inconsistencies. The only fix here is to add support for all these properties in NetworkManager as well. However, it's less serious, because with this commit we support several of the more important properties. See also the related bug rh#1337855 for kernel. Another difficulty is that `ip route replace` and `ip route change` changes an existing route. The replaced route has the same NM_PLATFORM_IP_ROUTE_CMP_TYPE_WEAK_ID, but differ in the actual NM_PLATFORM_IP_ROUTE_CMP_TYPE_ID: # ip -d -4 route show dev v # ip monitor route & # ip route add 192.168.5.0/24 dev v 192.168.5.0/24 dev v scope link # ip route change 192.168.5.0/24 dev v scope 10 192.168.5.0/24 dev v scope 10 # ip -d -4 route show dev v unicast 192.168.5.0/24 proto boot scope 10 Note that we only got one RTM_NEWROUTE message, although from NMPCache's point of view, a new route (with a particular ID) was added and another route (with a different ID) was deleted. The cumbersome workaround is, to keep an ordered list of the routes, and figure out which route was replaced in response to an RTM_NEWROUTE. In absence of bugs, this should work fine. However, as we only rely on events, we might wrongly introduce a cache-inconsistancy as well. See the related bug rh#1337860. Also drop nm_platform_ip4_route_get() and the like. The ID of routes is complex, so it makes little sense to look up a route directly.
2017-08-02 07:55:05 +02:00
static inline const NMDedupMultiEntry *
nm_dedup_multi_head_entry_get_idx (const NMDedupMultiHeadEntry *head_entry,
int idx)
{
CList *iter;
if (head_entry) {
if (idx >= 0) {
c_list_for_each (iter, &head_entry->lst_entries_head) {
if (idx-- == 0)
return c_list_entry (iter, NMDedupMultiEntry, lst_entries);
}
} else {
for (iter = head_entry->lst_entries_head.prev;
iter != &head_entry->lst_entries_head;
iter = iter->prev) {
if (++idx == 0)
return c_list_entry (iter, NMDedupMultiEntry, lst_entries);
}
}
}
return NULL;
}
core: track addresses for NMIP4Config/NMIP6Config via NMDedupMultiIndex Reasons: - it adds an O(1) lookup index for accessing NMIPxConfig's addresses. Hence, operations like merge/intersect have now runtime O(n) instead of O(n^2). Arguably, we expect low numbers of addresses in general. For low numbers, the O(n^2) doesn't matter and quite likely in those cases the previous implementation was just fine -- maybe even faster. But the simple case works fine either way. It's important to scale well in the exceptional case. - the tracked objects can be shared between the various NMPI4Config, NMIP6Config instances with NMPlatform and everybody else. - the NMPObject can be treated generically, meaning it enables code to handle both IPv4 and IPv6, or addresses and routes. See for example _nm_ip_config_add_obj(). - I want core to evolve to somewhere where we don't keep copies of NMPlatformIP4Address, et al. instances. Instead they shall all be shared. I hope this will reduce memory consumption (although tracking a reference consumes some memory too). Also, it shortcuts nmp_object_equal() when comparing the same object. Calling nmp_object_equal() on the identical objects would be a common case after the hash function pre-evaluates equality.
2017-07-07 23:34:41 +02:00
static inline void
nm_dedup_multi_head_entry_sort (const NMDedupMultiHeadEntry *head_entry,
CListSortCmp cmp,
gconstpointer user_data)
{
if (head_entry) {
/* the head entry can be sorted directly without messing up the
* index to which it belongs. Of course, this does mess up any
* NMDedupMultiIter instances. */
c_list_sort ((CList *) &head_entry->lst_entries_head, cmp, user_data);
}
}
shared: add nm_dedup_multi_entry_reorder() function This allows to reorder elements in NMDedupMultiIndex.
2017-08-05 10:26:17 +02:00
gboolean nm_dedup_multi_entry_reorder (const NMDedupMultiEntry *entry,
const NMDedupMultiEntry *entry_order,
gboolean order_after);
shared: add NMDedupMultiIndex "nm-dedup-multi.h" Add the NMDedupMultiIndex cache. It basically tracks objects as doubly linked list. With the addition that each object and the list head is indexed by a hash table. Also, it supports tracking multiple distinct lists, all indexed by the idx-type instance. It also deduplicates the tracked objects and shares them. - the objects that can be put into the cache must be immutable and ref-counted. That is, the cache will deduplicate them and share the reference. Also, as these objects are immutable and ref-counted, it is safe that users outside the cache own them too (as long as they keep them immutable and manage their reference properly). The deduplication uses obj_id_hash_func() and obj_id_equal_func(). These functions must cover *every* aspect of the objects when comparing equality. For example nm_platform_ip4_route_cmp() would be a function that qualifies as obj_id_equal_func(). The cache creates references to the objects as needed and gives them back. This happens via obj_get_ref() and obj_put_ref(). Note that obj_get_ref() is free to create a new object, for example to convert a stack-allocated object to a (ref-counted) heap allocated one. The deduplication process creates NMDedupIndexBox instances which are the ref-counted entity. In principle, the objects themself don't need to be ref-counted as that is handled by the boxing instance. - The cache doesn't only do deduplication. It is a multi-index, meaning, callers add objects using a index handle NMDedupMultiIdxType. The NMDedupMultiIdxType instance is the access handle to lookup the list and objects inside the cache. Note that the idx-type instance may partition the objects in distinct lists. For all operations there are cross-references and hash table lookups. Hence, every operation of this data structure is O(1) and the memory overhead for an index tracking an object is constant. The cache preserves ordering (due to linked list) and exposes the list as public API. This allows users to iterate the list without any additional copying of elements.
2017-06-04 22:43:21 +02:00
/*****************************************************************************/
#endif /* __NM_DEDUP_MULTI_H__ */
Reference in a new issue Copy permalink