<commit_before>
//!
//! # Purpose
//!
//! This module contains the functions exported for use by other languages.
//!
//! If you're writing bindings, you're in the right place. Don't use imageflow_core::ffi, that's
//! *not* correct - those are unstable and will disappear as C is replaced with Rust.
//!
//! # Memory Lifetimes
//!
//! In order to prevent dangling pointers, we must be correct about memory lifetimes.
//!
//! ## ... when allocated by Imageflow, assume the lifetime of the `context`
//!
//! **In Imageflow, by default, all things created with a context will be destroyed when the
//! context is destroyed.**
//!
//! This is very nice, as it means that a client's failure to clean up
//! will have limited impact on the process as a whole - as long as the client at minimum
//! calls `flow_context_destroy` at the end of all possible code paths.
//!
//! However, waiting to free memory and run destructors until the context is destroyed is not ideal;
//! it increases our peak memory usage/needs and may cause operations
//! to fail that would otherwise succeed.
//!
//! There are two ways to mitigate this.
//!
//! 1. Schedule the destruction to occur earlier, using ownership.
//! 2. Invoke the corresponding destroy function when you're done with the thing.
//!
//! Only certain things may be owners: `context`, `job`, and `io` pointers. Setting a
//! 'shorter-lived' owner, like the job (vs. context) can help, but can be less effective
//! than directly invoking the destroy function as soon as it is possible to do so.
//!
//! ### Destroying things
//!
//! * An `imageflow_context` should ALWAYS be destroyed with `imageflow_context_destroy`
//! * ImageflowJsonResponse structures should be released with `imageflow_json_response_destroy`
//! *
//!
//! ## ... when allocated by the client, Imageflow only borrows it for the `invocation`
//!
//! **Imageflow assumes that, at minimum, all pointers that you provide to it will, at minimum,
//! remain valid for the duration of the API call.** We'll call this 'borrowing'. Imageflow is
//! just borrowing it for a bit; not taking ownership of the thing.
//!
//! This may seem obvious, but it is not, in fact, guaranteed by garbage-collected languages. They
//! are often oblivious to pointers, and cannot track what data is and is not referenced.
//! Therefore, we suggest that you ensure every allocation made (and handed to Imageflow) is
//! referenced *after* the imageflow API call, preferably in a way that will not be optimized away
//! at runtime. Many languages and FFI libraries offer a utility method just for this purpose.
//!
//! ## ... although Imageflow may borrow some strings for the life of the context, yet not own them.
//!
//! This happens for strings that are usually static constants, and unlikely to be allocated on
//! the heap anyway.
//!
//! * When an Imageflow API asks for a filename, function name, or error message, it will
//!   assume that those strings are pointers that (a) Imageflow is not
//!   responsible for freeing, and (b) will (at least) outlive the `context`.
//!
//! ## ... and it should be very clear when Imageflow is taking ownership of something you created!
//!
//! When Imageflow needs continued access to data that is NOT highly likely to be static, it
//! will be documented.
//!
//! * If you give Imageflow a buffer to read an image from, it will need to access that buffer
//!   much longer than the initial io_create call.
//!
//! ## What if I need something to outlive the `context`?
//!
//! Then you'll need to change the owner - disassociate the thing from the context
//! , and become responsible for it,
//! and all the things it might have owned, all the destructors that will now never run.
//!
//! [TODO] Provide instructions
//!
//!
//! # Data types
//!
//! Reference for those creating bindings in other languages
//!
//! Two types are platform-specific - use the corresponding pointer or size type that varies with
//! your platform.
//!
//! * libc::c_void (or anything *mut or *const): Platform-sized pointer. 32 or 64 bits.
//! * The above includes *mut Context, *mut Job, *mut JobIO, etc.
//! * libc::size_t (or usize): Unsigned integer, platform-sized. 32 or 64 bits.
//!
//!
//! Treat *mut Context, *mut Job, *mut JobIO, *mut ImageflowJsonResponse ALL as opaque pointers.
//!
//!
//! Fixed size
//!
//! * u8 (1 unsigned byte)
//! * bool (C99 style, 1 byte, value 0 or 1)
//! * The rest seem self-explanatory.
//! * `i` prefixes signed ints
//! * `u` prefixes unsigned ints.
//! * `f` prefixes floating point
//!
//! Structs
//!
//! Consider all structures to be opaque. Do not attempt to access fields by offsets; rather,
//! use the accessor functions provided.
//!
//!
//!
//!
//!
//!
//!
//!

#![crate_type = "cdylib"]
extern crate libc;

extern crate imageflow_core;<commit_msg>libimageflowrs should alloc_system always<commit_after>
//!
//! # Purpose
//!
//! This module contains the functions exported for use by other languages.
//!
//! If you're writing bindings, you're in the right place. Don't use imageflow_core::ffi, that's
//! *not* correct - those are unstable and will disappear as C is replaced with Rust.
//!
//! # Memory Lifetimes
//!
//! In order to prevent dangling pointers, we must be correct about memory lifetimes.
//!
//! ## ... when allocated by Imageflow, assume the lifetime of the `context`
//!
//! **In Imageflow, by default, all things created with a context will be destroyed when the
//! context is destroyed.**
//!
//! This is very nice, as it means that a client's failure to clean up
//! will have limited impact on the process as a whole - as long as the client at minimum
//! calls `flow_context_destroy` at the end of all possible code paths.
//!
//! However, waiting to free memory and run destructors until the context is destroyed is not ideal;
//! it increases our peak memory usage/needs and may cause operations
//! to fail that would otherwise succeed.
//!
//! There are two ways to mitigate this.
//!
//! 1. Schedule the destruction to occur earlier, using ownership.
//! 2. Invoke the corresponding destroy function when you're done with the thing.
//!
//! Only certain things may be owners: `context`, `job`, and `io` pointers. Setting a
//! 'shorter-lived' owner, like the job (vs. context) can help, but can be less effective
//! than directly invoking the destroy function as soon as it is possible to do so.
//!
//! ### Destroying things
//!
//! * An `imageflow_context` should ALWAYS be destroyed with `imageflow_context_destroy`
//! * ImageflowJsonResponse structures should be released with `imageflow_json_response_destroy`
//! *
//!
//! ## ... when allocated by the client, Imageflow only borrows it for the `invocation`
//!
//! **Imageflow assumes that, at minimum, all pointers that you provide to it will, at minimum,
//! remain valid for the duration of the API call.** We'll call this 'borrowing'. Imageflow is
//! just borrowing it for a bit; not taking ownership of the thing.
//!
//! This may seem obvious, but it is not, in fact, guaranteed by garbage-collected languages. They
//! are often oblivious to pointers, and cannot track what data is and is not referenced.
//! Therefore, we suggest that you ensure every allocation made (and handed to Imageflow) is
//! referenced *after* the imageflow API call, preferably in a way that will not be optimized away
//! at runtime. Many languages and FFI libraries offer a utility method just for this purpose.
//!
//! ## ... although Imageflow may borrow some strings for the life of the context, yet not own them.
//!
//! This happens for strings that are usually static constants, and unlikely to be allocated on
//! the heap anyway.
//!
//! * When an Imageflow API asks for a filename, function name, or error message, it will
//!   assume that those strings are pointers that (a) Imageflow is not
//!   responsible for freeing, and (b) will (at least) outlive the `context`.
//!
//! ## ... and it should be very clear when Imageflow is taking ownership of something you created!
//!
//! When Imageflow needs continued access to data that is NOT highly likely to be static, it
//! will be documented.
//!
//! * If you give Imageflow a buffer to read an image from, it will need to access that buffer
//!   much longer than the initial io_create call.
//!
//! ## What if I need something to outlive the `context`?
//!
//! Then you'll need to change the owner - disassociate the thing from the context
//! , and become responsible for it,
//! and all the things it might have owned, all the destructors that will now never run.
//!
//! [TODO] Provide instructions
//!
//!
//! # Data types
//!
//! Reference for those creating bindings in other languages
//!
//! Two types are platform-specific - use the corresponding pointer or size type that varies with
//! your platform.
//!
//! * libc::c_void (or anything *mut or *const): Platform-sized pointer. 32 or 64 bits.
//! * The above includes *mut Context, *mut Job, *mut JobIO, etc.
//! * libc::size_t (or usize): Unsigned integer, platform-sized. 32 or 64 bits.
//!
//!
//! Treat *mut Context, *mut Job, *mut JobIO, *mut ImageflowJsonResponse ALL as opaque pointers.
//!
//!
//! Fixed size
//!
//! * u8 (1 unsigned byte)
//! * bool (C99 style, 1 byte, value 0 or 1)
//! * The rest seem self-explanatory.
//! * `i` prefixes signed ints
//! * `u` prefixes unsigned ints.
//! * `f` prefixes floating point
//!
//! Structs
//!
//! Consider all structures to be opaque. Do not attempt to access fields by offsets; rather,
//! use the accessor functions provided.
//!
//!
//!
//!
//!
//!
//!
//!

#![crate_type = "cdylib"]
#![feature(alloc_system)]

extern crate libc;
extern crate alloc_system;
extern crate imageflow_core;
{'fraction_non_alphanumeric': 0.12128381853718753, 'fraction_numerical': 0.003497582553235264, 'mean_word_length': 3.313220940550133, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2822395', 'n_tokens_mistral': 2838, 'n_tokens_neox': 2497, 'n_words': 1366}