jacobkahn · GitHub

@jacobkahn mentioned this pull request

Mar 19, 2019

Closed

Open

- Rename af_release_event to af_delete_event and make arg non-const
- Make af::event::block() const
- Improvements to memory manager API documentation throughout

@jacobkahn

@umar456

@umar456

- C API functions with the default memory manager were, when used with the unified backend, causing symbol table lookups which slowed things down
- A unified neural network example now benchmarks similarly on master and with custon memory manager integrations after the change when build and linked to the unified backend

@umar456

@umar456

facebook-github-bot pushed a commit to flashlight/flashlight that referenced this pull request

Mar 31, 2020
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

jacobkahn added a commit to flashlight/text that referenced this pull request

Mar 25, 2022
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

jacobkahn added a commit to flashlight/text that referenced this pull request

Mar 25, 2022
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

jacobkahn added a commit to flashlight/text that referenced this pull request

Mar 25, 2022
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

jacobkahn added a commit to flashlight/text that referenced this pull request

Mar 25, 2022
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

jacobkahn added a commit to flashlight/text that referenced this pull request

Mar 25, 2022
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

jacobkahn added a commit to flashlight/text that referenced this pull request

Mar 25, 2022
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

jacobkahn added a commit to flashlight/text that referenced this pull request

Mar 27, 2022
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

jacobkahn added a commit to flashlight/text that referenced this pull request

Mar 27, 2022
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

jacobkahn added a commit to flashlight/sequence that referenced this pull request

Oct 13, 2022
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

jacobkahn added a commit to flashlight/sequence that referenced this pull request

Oct 13, 2022
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

jacobkahn added a commit to flashlight/sequence that referenced this pull request

Oct 13, 2022
Summary:
Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`.
The framework has several components:
**A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461.
- Contains AF public interface functions that can be overriden to facilitate building a custom memory manager.
- Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff.
- `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc).
**A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions.
- AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation
- The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager`
- When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following:
  - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called.
  - Calls `log(...)` on the handle payload to log the native AF call
**A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior.
- To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`.
- `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl.
- Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions.
Reviewed By: avidov
Differential Revision: D19056964
fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01

Read the original on github.com ↗