Here it is an interesting case of cardinality feedback collected from an 11.2.0.3 running system. A simple query against a single table has a perfect first execution response time with, according to the human eyes, a quite acceptable difference between Oracle cardinality estimates and actual rows as shown below:
SELECT
tr_id
FROM
t1 t1
WHERE
t1.t1_col_name= 'GroupID'
AND t1.t1_col_value= '6276931'
AND EXISTS(SELECT
1
FROM
t1 t2
WHERE t1.tr_id = t2.tr_id
AND t2.t1_col_name= 'TrRangeOrder'
AND t2.t1_col_value= 'TrOrderPlace'
);
SQL_ID 8b3tv5uh8ckfb, child number 0
-------------------------------------
Plan hash value: 1066392926
--------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time |
--------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1 |00:00:00.14 |
| 1 | NESTED LOOPS SEMI | | 1 | 1 | 1 |00:00:00.14 |
| 2 | TABLE ACCESS BY INDEX ROWID| T1 | 1 | 1 | 6 |00:00:00.07 |
|* 3 | INDEX RANGE SCAN | IDX_T1_NAME_VALUE | 1 | 1 | 6 |00:00:00.03 |
|* 4 | TABLE ACCESS BY INDEX ROWID| T1 | 6 | 1 | 1 |00:00:00.07 |
|* 5 | INDEX UNIQUE SCAN | T1_PK | 6 | 1 | 6 |00:00:00.07 |
--------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("T1"."T1_COL_NAME"='GroupID' AND "T1"."T1_COL_VALUE"='6276931')
4 - filter("T2"."T1_COL_VALUE"='TrOrderPlace')
5 - access("T1"."TR_ID"="T2"."TR_ID" AND
"T2"."T1_COL_NAME"='TrRangeOrder')
And here it is the second dramatic execution plan and response time due, this time, to cardinality feedback optimisation:
SQL_ID 8b3tv5uh8ckfb, child number 1
-------------------------------------
Plan hash value: 3786385867
----------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time |
----------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1 |00:10:40.14 |
| 1 | NESTED LOOPS | | 1 | | 1 |00:10:40.14 |
| 2 | NESTED LOOPS | | 1 | 1 | 787K|00:09:31.00 |
| 3 | SORT UNIQUE | | 1 | 1 | 787K|00:02:44.83 |
| 4 | TABLE ACCESS BY INDEX ROWID| T1 | 1 | 1 | 787K|00:02:41.58 |
|* 5 | INDEX RANGE SCAN | IDX_T1_NAME_VALUE | 1 | 1 | 787K|00:00:36.46 |
|* 6 | INDEX UNIQUE SCAN | T1_PK | 787K| 1 | 787K|00:06:45.25 |
|* 7 | TABLE ACCESS BY INDEX ROWID | T1 | 787K| 1 | 1 |00:05:00.24 |
----------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
5 - access("T2"."T1_COL_NAME"='TrRangeOrder' AND "T2"."T1_COL_VALUE"='TrOrderPlace')
6 - access("T1"."TR_ID"="T2"."TR_ID" AND "T1"."T1_COL_NAME"='GroupID')
7 - filter("T1"."T1_COL_VALUE"='6276931')
Note
-----
- cardinality feedback used for this statement
There is no real noticeable difference between actual and estimated rows in the first run of the query (E-Rows =1 versus A-Rows = 6) which implies a new re-optimisation. But Oracle did it and marked the child cursor n°0 candidate for a cardinality feedback:
SQL> select
sql_id
,child_number
,use_feedback_stats
from
v$sql_shared_cursor
where
sql_id = '8b3tv5uh8ckfb';
SQL_ID CHILD_NUMBER U
------------- ------------ -
8b3tv5uh8ckfb 0 Y
The bad news however with this Oracle decision is that we went from a quasi-instantaneous response time to a catastrophic 10 min. In the first plan the always suspicious estimated ‘’1’’ cardinality is not significantly far from actual rows (6), so why then Oracle has decided to re-optimize the first cursor? It might be “possible” that when Oracle rounds up its cardinality estimation to 1 for a cursor that has been previously monitored for cardinality feedback, it flags somewhere that this cursor is subject to a re-optimization during its next execution whatever the actual rows will be (close to 1 or not)?
Fortunately, this second execution has also been marked for re-optimisation:
SQL> select
sql_id
,child_number
,use_feedback_stats
from
v$sql_shared_cursor
where
sql_id = '8b3tv5uh8ckfb';
SQL_ID CHILD_NUMBER U
------------- ------------ -
8b3tv5uh8ckfb 0 Y
8b3tv5uh8ckfb 1 Y
And the third execution of the query produces the following interesting execution plan
SQL_ID 8b3tv5uh8ckfb, child number 2
-------------------------------------
Plan hash value: 1066392926
--------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time |
--------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1 |00:00:00.01 |
| 1 | NESTED LOOPS SEMI | | 1 | 1 | 1 |00:00:00.01 |
| 2 | TABLE ACCESS BY INDEX ROWID| T1 | 1 | 6 | 6 |00:00:00.01 |
|* 3 | INDEX RANGE SCAN | IDX_T1_NAME_VALUE | 1 | 6 | 6 |00:00:00.01 |
|* 4 | TABLE ACCESS BY INDEX ROWID| T1 | 6 | 1 | 1 |00:00:00.01 |
|* 5 | INDEX UNIQUE SCAN | T1_PK | 6 | 1 | 6 |00:00:00.01 |
--------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("T1"."T1_COL_NAME"='GroupID' AND "T1"."T1_COL_VALUE"='6276931')
4 - filter("T2"."T1_COL_VALUE"='TrOrderPlace')
5 - access("T1"."TR_ID"="T2"."TR_ID" AND
"T2"."T1_COL_NAME"='TrRangeOrder')
Note
-----
- cardinality feedback used for this statement
Oracle is back to its first execution plan. The new estimations coincide perfectly with the actuals so that Oracle decided to stop monitoring this cursor with cardinality feedback as shown below:
SQL> select
sql_id
,child_number
,use_feedback_stats
from
v$sql_shared_cursor
where
sql_id = '8b3tv5uh8ckfb';
SQL_ID CHILD_NUMBER U
------------- ------------ -
8b3tv5uh8ckfb 0 Y
8b3tv5uh8ckfb 1 Y
8b3tv5uh8ckfb 2 N
Several questions come to my mind at this stage of the investigation:
- What are the circumstances for which Oracle marks a cursor for cardinality feedback optimisation?
- How Oracle decides that E-Rows are significantly different from A-Rows and henceforth a cursor re-optimization will be done? In other words is E-Rows =1 significantly different from A-Rows =6? Or does that suspicious cardinality 1 participate in Oracle decision to re-optimize a cursor monitored with cardinality feedback?
Let’s try to answer the first question. There is only one unique table involved in this query with two conjunctive predicates. The two predicate columns have the following statistics
SQL> select
column_name
,num_distinct
,density
,histogram
from
all_tab_col_statistics
where
table_name = 'T1'
and
column_name in ('T1_COL_NAME','T1_COL_VALUE');
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM
--------------- ------------ ---------- ---------------
T1_COL_NAME 103 4,9781E-09 FREQUENCY
T1_COL_VALUE 14833664 ,000993049 HEIGHT BALANCED
The presence of histograms, particularly the HEIGHT BALANCED, on these two columns participates strongly in the Oracle decision to monitor the cursor for cardinality feedback. In order to be sure of it I decided to get rid of histograms from both columns and re-query again:
SQL> select
column_name
,num_distinct
,density
,histogram
from
all_tab_col_statistics
where
table_name = 'T1'
and
column_name in ('T1_COL_NAME','T1_COL_VALUE');
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM
--------------- ------------ ---------- ---------
T1_COL_NAME 103 ,009708738 NONE
T1_COL_VALUE 15477760 6,4609E-08 NONE
The new cursor is not anymore monitored with the cardinality feedback as shown below:
SQL_ID fakc7vfbu1mam, child number 0
-------------------------------------
Plan hash value: 739349168
--------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time |
--------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1 |00:02:00.68 |
|* 1 | HASH JOIN SEMI | | 1 | 6 | 1 |00:02:00.68 |
| 2 | TABLE ACCESS BY INDEX ROWID| T1 | 1 | 6 | 6 |00:00:00.01 |
|* 3 | INDEX RANGE SCAN | IDX_T1_NAME_VALUE | 1 | 6 | 6 |00:00:00.01 |
| 4 | TABLE ACCESS BY INDEX ROWID| T1 | 1 | 6 | 787K|00:02:00.14 |
|* 5 | INDEX RANGE SCAN | IDX_T1_NAME_VALUE | 1 | 6 | 787K|00:00:12.36 |
--------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("T1"."TR_ID"="T2"."TR_ID")
3 - access("T1"."T1_COL_NAME"='GroupID' AND "T1"."T1_COL_VALUE"='6276931')
5 - access("T2"."T1_COL_NAME"='TrRangeOrder' AND "T2"."T1_COL_VALUE"='TrOrderPlace')
SQL> select
sql_id
,child_number
,use_feedback_stats
from
v$sql_shared_cursor
where
sql_id = 'fakc7vfbu1mam';
SQL_ID CHILD_NUMBER U
------------- ------------ -
fakc7vfbu1mam 0 N --> cursor not re-optimisable
Without histograms on the two columns Oracle has not monitored the query for cardinality feedback. Unfortunately getting rid of histogram was not an option accepted by the client nor changing this packaged query to force the optimizer not unnesting the EXISTS subquery with its parent query as far as the later is always generating a couple of rows that will not hurt performance when filtered with the EXIST subquery. Attaching a SQL Profile has also been discarded because several copies of the same query are found in the packaged application which would have necessitated a couple of extra SQL Profiles.
The last option that remains at my hands was to collect extended statistics so that Oracle will be able to get accurate estimations and henceforth will stop using cardinality feedback
SQL> SELECT
dbms_stats.create_extended_stats
(ownname => user
,tabname => 'T1'
,extension => '(T1_COL_NAME,T1_COL_VALUE)'
)
FROM dual;
DBMS_STATS.CREATE_EXTENDED_STATS(
---------------------------------
SYS_STUE3EBVNLB6M1SYS3A07$LD52
SQL> begin
dbms_stats.gather_table_stats
(user
,'T1'
,method_opt => 'for columns SYS_STUE3EBVNLB6M1SYS3A07$LD52 size skewonly'
,cascade => true
,no_invalidate => false
);
end;
/
SQL> select
column_name
,num_distinct
,density
,histogram
from all_tab_col_statistics
where
table_name = 'T1'
and column_name in ('T1_COL_NAME','T1_COL_VALUE', 'SYS_STUE3EBVNLB6M1SYS3A07$LD52');
COLUMN_NAME NUM_DISTINCT DENSITY HISTOGRAM
------------------------------ ------------ ---------- ---------------
SYS_STUE3EBVNLB6M1SYS3A07$LD52 18057216 ,000778816 HEIGHT BALANCED
T1_COL_NAME 103 4,9781E-09 FREQUENCY
T1_COL_VALUE 14833664 ,000993049 HEIGHT BALANCED
SQL_ID dn6p58b9b6348, child number 0
-------------------------------------
Plan hash value: 1066392926
--------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time |
--------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1 |00:00:00.01 |
| 1 | NESTED LOOPS SEMI | | 1 | 3 | 1 |00:00:00.01 |
| 2 | TABLE ACCESS BY INDEX ROWID| T1 | 1 | 3 | 6 |00:00:00.01 |
|* 3 | INDEX RANGE SCAN | IDX_T1_NAME_VALUE | 1 | 3 | 6 |00:00:00.01 |
|* 4 | TABLE ACCESS BY INDEX ROWID| T1 | 6 | 832K| 1 |00:00:00.01 |
|* 5 | INDEX UNIQUE SCAN | T1_PK | 6 | 1 | 6 |00:00:00.01 |
--------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("T1"."T1_COL_NAME"='GroupID' AND "T1"."T1_COL_VALUE"='6276931')
4 - filter("T2"."T1_COL_VALUE"='TrOrderPlace')
5 - access("T1"."TR_ID"="T2"."TR_ID" AND
"T2"."T1_COL_NAME"='TrRangeOrder')
SQL> select
sql_id
,child_number
,use_feedback_stats
from
v$sql_shared_cursor
where
sql_id = 'dn6p58b9b6348';
SQL_ID CHILD_NUMBER U
------------- ------------ -
dn6p58b9b6348 0 N
This time, for E-Rows = 3 and A-Rows =6, Oracle decided that there is no significant difference between cardinality estimates and the actual rows so that the cursor is not anymore subject to cardinality feedback optimization.
You might have pointed out that I have forced the Extended Statistics column to have histogram. Otherwise the cardinality feedback will kicks off. In fact I have conducted several experiments to see when the cardinality feedback occurs and when not depending on the existence or the absence of the column group extension, its type of statistics and the statistics that have been gathered on the underlying two columns predicates:
