Tensors allocated on cache does not appear deallocation capable

[jerinphilip]

Assuming TensorAllocator is an attempt at implementing an equivalent of std::allocator.

Both tensors_ and cache_ in ExpressionGraph are TensorAllocators.

The cache_ allocates some tensors that are incapable of being deallocated from outside. cache_ is private to this class, and the missing free is a flaw(?).

marian-dev/src/graph/expression_graph.h

Lines 57 to 64 in 844800e

	if(!node->val()) {
	if(node->memoize())
	cache_->allocate(node->val(), node->shape(), node->value_type());
	else
	tensors_->allocate(node->val(), node->shape(), node->value_type());
	}
	}

I could not find a cache_->free(), tensors_ on the other hand are capable of being freed manually through external calls.

marian-dev/src/graph/expression_graph.h

Line 70 in 844800e

void free(const Tensor& tensor) { tensors_->free(tensor); }

[jerinphilip]

/marian-dev/src$ grep -R "setMemoize([^)]*)" --exclude-dir 3rd_party/
tensors/cpu/intgemm_interface.h:    setMemoize(false);
tensors/cpu/intgemm_interface.h:    setMemoize(false); // Enabling memoization leads to a massive memory leak. Instead use special "midterm" memory.
tensors/cpu/intgemm_interface.h:                       // Still, I don't understand why setMemoize(true) still leaks.
tensors/cpu/intgemm_interface.h:      setMemoize(false);
tensors/cpu/intgemm_interface.h:      setMemoize(false);
tensors/cpu/intgemm_interface.h:      setMemoize(false);
tensors/cpu/intgemm_interface.h:        setMemoize(false); // AFAIK dot is never called with the same matrices
tensors/cpu/intgemm_interface.h:        setMemoize(false); // AFAIK affine is never called with the same matrices
graph/node_operators.cpp:  setMemoize(graph->isInference());
graph/node.h:  virtual void setMemoize(bool memoize) override { memoize_ = memoize; };
graph/chainable.h:  virtual void setMemoize(bool) = 0;

Memoization appears to be set for nodes at inference.

[jerinphilip]

@XapaJIaMnu

Still, I don't understand why setMemoize(true) still leaks.