<jupyter_start><jupyter_text>OpEn Rust Advanced Example Test This example solves a simple optimisation problem (advanced example) by using [OpEn's rust interface](https://alphaville.github.io/optimization-engine/docs/openrust-alm).This example is just a duplicate of [this tutorial](https://alphaville.github.io/optimization-engine/docs/openrust-alm), just for a practice purpose.   Features- This example solves more general problems that involve constraints of the general form $F_1(u) \in C$ and $F_2(u) = 0$. - This uses ALM/PM solver  Implementation First, we need to import "optimization_engine"- In your local PC, it should be also declared in "Cargo.toml".- In this jupyter notebook, we need to have "extern crate" as follows.<jupyter_code>extern crate optimization_engine;
use optimization_engine::{
    alm::*,
    constraints::*, panoc::*, *
};<jupyter_output><empty_output><jupyter_text>Objective FunctionSuppose that$$f(u) = \frac{1}{2} \| u \|^2 + u_1 + u_2 + u_3$$with gradient$$\nabla f(u) = u + 1_3 $$<jupyter_code>pub fn f(u: &[f64], cost: &mut f64) -> Result<(), SolverError> {
    *cost = 0.5 * matrix_operations::norm2_squared(u) + matrix_operations::sum(u);
    Ok(())
}

pub fn df(u: &[f64], grad: &mut [f64]) -> Result<(), SolverError> {
    grad.iter_mut()
        .zip(u.iter())
        .for_each(|(grad_i, u_i)| *grad_i = u_i + 1.0);
    Ok(())
}<jupyter_output><empty_output><jupyter_text>ConstraintsWe impose a constraint:$$F_1(u) \in C$$where $$F_1(u) = \begin{bmatrix}2 u_1 + u_3 + 0.5 \\u_1 + 3u_2 \end{bmatrix}$$$$C = \{z \in \mathbb{R}^2: \|z\| \le 1 \}$$<jupyter_code>pub fn f1(u: &[f64], f1u: &mut [f64]) -> Result<(), SolverError> {
    assert!(u.len() == 3, "the length of u must be equal to 3");
    assert!(f1u.len() == 2, "the length of F1(u) must be equal to 2");
    f1u[0] = 2.0 * u[0] + u[2] + 0.5;
    f1u[1] = u[0] + 3.0 * u[1];
    Ok(())
}<jupyter_output><empty_output><jupyter_text>We need to define Jacobian product (I don't understand it yet. I think it is necessary to use the ALM/PM optimisation method). $$JF_1^{\top} \cdot d = \begin{bmatrix}2d_1 + d_2 \\3d_2 \\d_1\end{bmatrix}$$where $J F_1$ is the Jacobian matrix of $F_1$ for given vectors $u \in \mathbb{R}^3$ (decision variables) and $d \in \mathbb{R}^2$ (dimension of constraints?).NOTE:$$JF_1 = \begin{bmatrix}\frac{\partial F_{1,1}}{\partial u_1} & \frac{\partial F_{1,1}}{\partial u_2} & \frac{\partial F_{1,1}}{\partial u_3} \\\frac{\partial F_{1,2}}{\partial u_1} & \frac{\partial F_{1,2}}{\partial u_2} & \frac{\partial F_{1,2}}{\partial u_3} \end{bmatrix}= \begin{bmatrix}2 & 0 & 1 \\1 & 3 & 0\end{bmatrix}$$<jupyter_code>pub fn f1_jacobian_product(_u: &[f64], d: &[f64], res: &mut [f64]) -> Result<(), SolverError> {
    assert!(d.len() == 2, "the length of d must be equal to 3");
    assert!(res.len() == 3, "the length of res must be equal to 3");
    res[0] = 2.0 * d[0] + d[1];
    res[1] = 3.0 * d[1];
    res[2] = d[0];
    Ok(())
}<jupyter_output><empty_output><jupyter_text>Now, the problem we need to solve is$$\text{Minimise} f(u)$$$$\text{subject to } -\frac{1}{2} \le u_i \le \frac{1}{2} \text{  for all $i$}$$$$F_1(u,p) \in C$$  Invoking the main function `AlmFactory` - construct $\psi$ and its gradient- [Input structure](https://docs.rs/optimization_engine/0.6.2/src/optimization_engine/alm/alm_factory.rs.html150-186) shoud be```pub fn new(    f: Cost,    df: CostGradient,    mapping_f1: Option,    jacobian_mapping_f1_trans: Option,    mapping_f2: Option,    jacobian_mapping_f2_trans: Option,    set_c: Option,    n2: usize,)```<jupyter_code>fn main() {
    let tolerance = 1e-5;
    let nx = 3; // problem_size: dimension of the decision variables
    let n1 = 2; // range dimensions of mappings F1
    let n2 = 0; // range dimensions of mappings F2
    let lbfgs_mem = 3; // memory of the LBFGS buffer
    
    // PANOCCache: All the information needed at every step of the algorithm
    let panoc_cache = PANOCCache::new(nx, tolerance, lbfgs_mem);
    
    // AlmCache: A cache structure that contains all the data 
    // that make up the state of the ALM/PM algorithm
    // (i.e., all those data that the algorithm updates)
    let mut alm_cache = AlmCache::new(panoc_cache, n1, n2);

    let set_c = Ball2::new(None, 0.5);
    let bounds = Ball2::new(None, 10.0);
    let set_y = Ball2::new(None, 1e12);

    // AlmFactory: Prepare function psi and its gradient 
    // given the problem data such as f, del_f and 
    // optionally F_1, JF_1, C, F_2
    let factory = AlmFactory::new(
        f, // Cost function
        df, // Cost Gradient
        Some(f1), // MappingF1
        Some(f1_jacobian_product), // Jacobian Mapping F1 Trans
        NO_MAPPING, // MappingF2
        NO_JACOBIAN_MAPPING, // Jacobian Mapping F2 Trans
        Some(set_c), // Constraint set
        n2,
    );

    // Define an optimisation problem 
    // to be solved with AlmOptimizer
    let alm_problem = AlmProblem::new(
        bounds,
        Some(set_c),
        Some(set_y),
        |u: &[f64], xi: &[f64], cost: &mut f64| -> Result<(), SolverError> {
            factory.psi(u, xi, cost)
        },
        |u: &[f64], xi: &[f64], grad: &mut [f64]| -> Result<(), SolverError> {
            factory.d_psi(u, xi, grad)
        },
        Some(f1),
        NO_MAPPING,
        n1,
        n2,
    );

    let mut alm_optimizer = AlmOptimizer::new(&mut alm_cache, alm_problem)
        .with_delta_tolerance(1e-5)
        .with_max_outer_iterations(20)
        .with_epsilon_tolerance(1e-6)
        .with_initial_inner_tolerance(1e-2)
        .with_inner_tolerance_update_factor(0.5)
        .with_initial_penalty(100.0)
        .with_penalty_update_factor(1.05)
        .with_sufficient_decrease_coefficient(0.2)
        .with_initial_lagrange_multipliers(&vec![5.0; n1]);

    let mut u = vec![0.0; nx];
    let solver_result = alm_optimizer.solve(&mut u);
    let r = solver_result.unwrap();
    println!("\n\nSolver result : {:#.7?}\n", r);
    println!("Solution u = {:#.6?}", u);
}<jupyter_output><empty_output><jupyter_text>Let's excute it.<jupyter_code>main();<jupyter_output>Solver result : AlmOptimizerStatus {
    exit_status: Converged,
    num_outer_iterations: 15,
    num_inner_iterations: 224,
    last_problem_norm_fpr: 0.0000005,
    lagrange_multipliers: Some(
        [
            -0.3072591,
            -0.3035106,
        ],
    ),
    solve_time: 116.8880000µs,
    penalty: 147.7455444,
    delta_y_norm: 0.0000116,
    f2_norm: 0.0000000,
}

Solution u = [
    -0.081619,
    -0.089919,
    -0.692479,
]
{'fraction_non_alphanumeric': 0.157269125266687, 'fraction_numerical': 0.04571776897287412, 'mean_word_length': 3.2232947232947233, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 36, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '110844', 'license': 'MIT', 'n_tokens_mistral': 2641, 'n_tokens_neox': 2450, 'n_words': 624, 'path': 'tools_examples/OpEn/examples_rust/OpEn_Rust_example_03_ALMPM.ipynb', 'repo_name': 'inmo-jang/optimisation', 'size': '6562'}