Closed
Open
- Rename af_release_event to af_delete_event and make arg non-const - Make af::event::block() const - Improvements to memory manager API documentation throughout
- C API functions with the default memory manager were, when used with the unified backend, causing symbol table lookups which slowed things down - A unified neural network example now benchmarks similarly on master and with custon memory manager integrations after the change when build and linked to the unified backend
facebook-github-bot pushed a commit to flashlight/flashlight that referenced this pull request
Mar 31, 2020Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01
jacobkahn added a commit to flashlight/text that referenced this pull request
Mar 25, 2022Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01
jacobkahn added a commit to flashlight/text that referenced this pull request
Mar 25, 2022Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01
jacobkahn added a commit to flashlight/text that referenced this pull request
Mar 25, 2022Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01
jacobkahn added a commit to flashlight/text that referenced this pull request
Mar 25, 2022Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01
jacobkahn added a commit to flashlight/text that referenced this pull request
Mar 25, 2022Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01
jacobkahn added a commit to flashlight/text that referenced this pull request
Mar 25, 2022Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01
jacobkahn added a commit to flashlight/text that referenced this pull request
Mar 27, 2022Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01
jacobkahn added a commit to flashlight/text that referenced this pull request
Mar 27, 2022Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01
jacobkahn added a commit to flashlight/sequence that referenced this pull request
Oct 13, 2022Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01
jacobkahn added a commit to flashlight/sequence that referenced this pull request
Oct 13, 2022Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01
jacobkahn added a commit to flashlight/sequence that referenced this pull request
Oct 13, 2022Summary: Beginning to check in some memory management framework code. **NB*: I won't land this until ArrayFire 3.7 is out; for now, this only runs on master. Putting in `contrib` for now since it doesn't build inside FB, but will move this once it's landable into `flashlight/flashlight/memory`. The framework has several components: **A C++ wrapper for the ArrayFire C memory manager API** (`fl::MemoryManagerAdapter`)added in arrayfire/arrayfire#2461. - Contains AF public interface functions that can be overriden to facilitate building a custom memory manager. - Everything else is totally unopinionated. Besides JIT functions, memory management implementations can theoretically do anything. I'm leaving out interoperability with `af_event` here and will add that in a separate diff. - `fl::MemoryManagerAdapter` differs slightly from the internal AF API because that needs to support opinionated memory functions that are also in the public AF device API (such as memory step size, `usageInfo`, etc). **A memory manager adaptor** to facilitate easily using C++ memory manager implementations/wrap AF functions. - AF memory management APIs expect an `af_memory_manager` for ABI compatibility, the manager adapter creates an `af_memory_manager` which corresponds to the AF handle corresponding to the C++ implementation - The `fl::MemoryManagerAdapter` that corresponds to the `af_memory_manager` is added as the payload to the `af_memory_manager` - When the manager installer is created and passed an `fl::MemoryManagerAdapter`, it creates function pointers and sets those in the relevant `af_memory_manager`. Each function pointer accomplishes the following: - Function pointer callbacks for the AF memory manager API are all passed an `af_memory_manager`. Since the C++ implementation is a `void*` paylaod on the `af_memory_manager`, it can be retrieved, then the proper C++ function on the implementation called. - Calls `log(...)` on the handle payload to log the native AF call **A logging framework for memory management** that logs ArrayFire requests for memory and calls to functions inspecting memory manager state used to determine JIT behavior. - To enable logging with a memory manager, call `fl::MemoryManagerAdapter::setLoggingEnabled(...)` after setting an output stream with `setLogStream`. - `fl::MemoryManagerAdapter::log(...)` is called inside the manager adapter's lambdas using the function on the impl. - Logging can also easily be performed from the memory manager directly using `log(...)` in order to log user-defined functions. Reviewed By: avidov Differential Revision: D19056964 fbshipit-source-id: b02e0107d9cfab2f09abbb5f55774b89679a6f01