Other ideas to consider

[dahong67]

• more careful consideration of types for init
• reduce allocs by adding buffer arg to mttkrps! and grad_U!
• consider moving default functions to corresponding modules, e.g., default_constraints(...) -> GCPConstraints.default(...)
• consider renaming losses to drop "Loss" suffix
• Support computation of non-Float64 decompositions (GPUArrays? Units?)
• Improvements to benchmarking mentioned in Add Benchmarks #30 (comment)
• Audit/optimize memory usage (avoid unnecessary allocations, etc.)
• Create / use AbstractCPD type and consider other representations
• Create abstract TensorFactorization type and define an interface
• Better / more complete implementation of getindex for CPD
• See if there are cases where it is faster to compute the MTTKRP column by column (to avoid forming full Khatri-Rao products which could be expensive when the rank is large)
• Improve docstrings for mttkrp and khatrirao, add more discussion / explanations / teaching (what exactly do they compute? what are some of the techniques used to make the implementations efficient?)
• Add tests for mttkrp and khatrirao specifically
• Have multiple algorithms for mttkrp and khatrirao (including old/simple/straightforward/naive implementations) and possibly put them in a separate TensorKernels module
• Better to store factor matrices by component rather than by mode?
• Better error messages!
• Implement iterative algorithms as iterators, see: https://iterativesolvers.julialinearalgebra.org/stable/iterators/
• Better default initialization - should probably depend on the loss/constraints/algorithm
• Use https://github.com/ericphanson/Asciicast.jl to make docs more animated
• convert methods (cf: https://github.com/JuliaLang/julia/blob/0e6d7972c62eb1dedaf00ea9b966a0060e2effb8/stdlib/LinearAlgebra/src/factorization.jl#L101-L104)
• faster version of Array (split into product of two khatri-rao's)
• more memory-efficient version of permutecomps! defined in terms of swaps, analogous to: https://github.com/JuliaLang/julia/blob/99d1d6723d72d43074b5d2765e63d08439989e8e/base/combinatorics.jl#L66-L95
• variant of permutecomps or sortcomps that uses a reference tensor as in: http://tensorly.org/stable/modules/generated/tensorly.cp_tensor.cp_permute_factors.html
• SubCPD type that provides a view into a CPD. Note: the CPD type actually already supports "views" in a sense since the λandUfields can themselves beSubArray`s, but this could be a bit confusing for new users. Ref: https://github.com/JuliaLang/julia/blob/48d4fd48430af58502699fdf3504b90589df3852/base/subarray.jl
• add getλ and getU functions. Ref: https://docs.julialang.org/en/v1/manual/style-guide/#Prefer-exported-methods-over-direct-field-access
• extension for itensors (https://github.com/ITensor/ITensors.jl)