OP Should Not Have Unused Input(English)

OP Specification: OP Should Not Have Unused Input

---

OP规范：OP中不允许注册未使用的Input (中文版)

---

Specification summary：

• Section 1, Background
• Section 2, Check method
• Section 3, Suggested fix of failed PR

Additional instructions:

1. During the implementation process, the specifications may find aspects that are not considered by the existing specifications, and need to be supplemented and improved in the implementation process. Please also give positive feedback.
2. Further analysis will be performed on the white list and the Operators will be refined if they can.

1. Background

Currently, the inputs of an Operator are registered by v in PaddlePaddle. However, there are several Operators which contains some inputs that is not used during the computation. It causes some problems,

• Waste of memory. PaddlePaddle's memory optimization strategy relies on precise life cycle analysis of Variable by reference count. If a Variable is registered as input of an Operator, its reference count increases by 1, which will extend the Variable's life and leads to waste of memory, especially GPU memory.
• Inconsistent of code. It may confuse readers if a Variable is registered as input but never used.

In order to avoid unused inputs of Operators, we proposed this OP Specification and add a corresponding check in CI, called UnusedVarCheck.

Check method

• Before running the Operator，initialize a ThreadLocal set to record the inputs used during computation.
• After running，compare the inputs of the Operator and recorded set, find the unused inputs.
  • Skip Operators that use mkldnn, since they adopt special memory optimization strategy.
  • Skip inputs that registered NoNeedBufferVarsInference, since they hold no memory buffer.
  • Skip inputs that is not initialized, since they hold no memory buffer.
• Use FLAGS_enable_unused_var_check to control the check. UnusedVarCheck is enabled in PR_CI_Coverage only.
• Use a white list to record the Operators which are failed in UnusedVarCheck, see details in unused_var_check.cc.

Suggested fix of failed PR

If an Operator is failed in UnusedVarCheck，build_log should contains message like：

Error Message Summary:
----------------------
PermissionDeniedError: Unused input variables check failed: Operator op_with_unused_var has input(s) not uesed: X, please make sure it(them) is(are) needed. 
If not, remove it(them) from inputs; if yes, register NoNeedBufferVars or add the operator to white list in unused_var_check.h.
  [Hint: Expected unsed_input_var_names.size() == 0, but received unsed_input_var_names.size():1 != 0:0.] at (/Paddle/Paddle/paddle/fluid/framework/unused_var_check.cc:82)

The three potential reasons of the failure and corresponding suggested fixes are,

• The dims and tensor of the input is never used during computation.
  • Fix: remove the input from OpMaker/GradOpMaker.
  • Example: abs_grad in PR24107.
• The dims the input is used during computation but the tensor is not.
  • Fix：register NoNeedBufferVarsInference for the input.
  • Example: flatten_grad in PR21169.
• The tensor of the input is used during computation in some cases（in if condition or in CUDA kernel but not in CPU kernel）.
  • Fix：add the Operator to the white list in unused_var_check.cc。
  • Example: fake_quantize_range_abs_max in PR21169, since FakeQuantizeRangeAbsMaxKernel uses different inputs in different if conditions.
  • Note: changing white list requires approval which will be described in the CI message.

If you have any problem, please feel free to contact @zhiqiu。