38 minutes ago, mychii said:

hen you choose you assume my instances will be all over 64 bytes

For instancess that less than 64 byte  better to make other mechanism then for fat instanes. Also it good idea to have a only external ref to such small instanses into owner object and perform any requests to its thru the its owner object that is same as using handles. Also it have sence to perform a defragmentations only in case pool iterated much often than elements accessed tru external refs for reconfiguring and so on, that usually applicable for create-and-forget instances processed by iterating a pool only.

I'm wondering. Is there anyone else here can help me or is my question wasn't clear enough? I have just simplified the question. I'd love to hear more answers from more minds.

I read the thread, but had trouble following parts of it. Maybe your question is perfectly clear as is, but stating it in a different way might make it easier for others to offer feedback.

Based on your original question at least, it's not immediately clear to me what role you want standard library containers to play. What do you want to store in the containers? The handles? Or the objects themselves? And what types of containers do you have in mind (vector, list, map, etc.)? Can you give an example (code or pseudocode) of the kind of operations you want to perform?

Anyway, if you're still looking for ideas, fleshing things out a bit further might help.

6 hours ago, Zakwayda said:

I read the thread, but had trouble following parts of it. Maybe your question is perfectly clear as is, but stating it in a different way might make it easier for others to offer feedback.

Thanks for attempting.

Quote

Based on your original question at least, it's not immediately clear to me what role you want standard library containers to play. What do you want to store in the containers? The handles? Or the objects themselves?

As long as it's not the object themselves directly (presented by new, or malloc + placement new, or whatever else similar returning direct pointer to object in memory). The reason has been stated that the object themselves don't always stay in the same location in the heap every certain point of time, which will cause the STL to possibly point to an invalid object.

What will be stored in the containers can be anything, in terms of elements, in terms of usage; such as int, float, Vector3, pointers, etc. The object handle (or memory/data/chunk handle, whatever you like to call it) is only giving the position where the array is in the memory. If you notice how std::vector works, if it changes capacity due to growth, all the elements in the vector returns a different address. That address changes because it reallocates its whole memory chunk to a different place in the heap, and that whole chunk is what my object handle is handling in this case.

Quote

And what types of containers do you have in mind (vector, list, map, etc.)?

Let's just say it's vector for now.

Quote

Can you give an example (code or pseudocode) of the kind of operations you want to perform?

Anyway, if you're still looking for ideas, fleshing things out a bit further might help.

The kind of operations are exactly just like a normal vector usage, it can be anything on gameplay level (inventory, list of npcs in current map, etc.).

On 10/3/2018 at 4:35 AM, mychii said:

The current best solutions are either modifying the containers, or make my own.

I too had trouble understanding what you were getting at, I think your issue is you want to use e.g. a vector, but have the memory it uses be relocatable.

I don't know STL super well internally as I tend to use my own template library these days (writing your own is quite doable, especially for POD types), but it hopefully wouldn't be a big deal to change the array start pointer yourself, even adding a public function for this. Whether you'd get any issues with parts of the STL code having dangling pointers, I don't know.

You could wrap all the references to the array start through an accessor function and see where it gets called, or override it to use your handle (which is a bit inefficient). You would have to be careful where and when you changed the array pointer, so there were no dangling pointers or maybe multithreaded code trying to access the vector at the same time.

Whether all this is the right solution for your problem is another matter. Usually I would think that a specific / alternative solution for your problem would tend to be a better bet, unless you are trying to create some kind of all singing, all dancing environment for others to use (e.g. multi-purpose engine / scripting language), and you have profiled this type of approach as being beneficial.

Bear in mind also that a lot of platforms these days offer virtual memory address space, so 'fragmentation' may not affect things in the same way as a flat address space.

Anyway thanks lawnjelly for replying, but I guess let me try again to clarify with picture since the replies have been clearly being troubled with what I'm getting at. It's probably the topic is a bit difficult (or weird) with addition me simply being so bad at explaining what I'm trying to get to or missing some (or using some bad) terms or even wrongly explained the underlying stuff. So please let me try this again, and please correct me if I'm wrong. This time with picture:

Hopefully this can clear things up to help you guys with less trouble... ?

The point of your handles is presumably to deal with a block of memory that may be relocatable. As far as I can see if you want to use a standard vector you either need to:

• Override the mechanism for determining the start of the vector array to go through the handle each time it is accessed (pretty inefficient) or
• Update the start of array pointer in the vector on relocating the memory (and watch for dangling pointers)

Afaik, underneath it all, although the implementation isn't specified, a vector is typically just a fancy dynamic array. If you want the vector to use your handle you would presumably have to provide it with the array address, either on the fly, or when it changes (preferable).

It may be an idea to write your own template wrapper if you intend to bandit an STL implementation, to emphasise it is working differently under the hood (as relocatable code may break the 'expected contract').

I must say I still strongly suspect this is something that could be achieved much better with a different paradigm, however, without knowing the details we can only answer what you have asked directly.

Actually thinking about it you may be able to force an allocation through your relocation code without modifying the STL code .. by clever use of reserve and resize. Depends on the STL implementation though and how it is about shrinking vectors.

i.e. You want to move array from A to B, array size is 10, max size is 16.

resize(10) according to this (http://www.cplusplus.com/reference/vector/vector/resize/) should result in reallocation enabling you to relocate in your custom allocator.

In the case where array size == max size, you might need to force an allocation with a resize(11) then delete the last element .. which might be costly depending on the object. There may be a better way, just an idea.

@mychii: Thanks for the helpful diagram. For me at least, the missing piece of the puzzle was that I didn't realize you wanted entire blocks of array memory to be relocatable (I thought that in some way you wanted to store multiple relocatable objects in the array, which I couldn't make sense of).

It's always possible I'm wrong, but I'm going to go out on a limb and say that I think it's unlikely there's a portable, reliable way to relocate the storage for (e.g.) a std::vector instance. (I'm happy to be corrected though.) If I'm right about that, then it seems like a custom container might be an appropriate solution.

lawnjelly's idea about forcing reallocations is interesting, but as he mentions the behavior would likely be implementation-dependent. In any case, I certainly wouldn't be comfortable relying on something like that.