Lucas Jellema (AMIS, The Netherlands)

THE VERY, VERY LATEST IN

ORACLE DATABASE DEVELOPMENT

Oracle Open World 2012, San Francisco

Public Expertezed Session Thursday 29th
November 2012
THE VERY VERY VERY LATEST

<Secret Code>
TOM KYTE TO THE RESCUE
THE DATABASE
IN MODERN ARCHITECTURES

Oracle Open World 2012, San Francisco

NO SQL
THE TOP-3 EARNING EMPLOYEES

What can you say about the result of this query

with respect to the question: Who are our top
three earning employees?

A. Correct Answer
B. Sometimes correct
C. Correct if there are never duplicate
salaries
D. Not Correct
IN-LINE VIEWS
TOP-N QUERIES IN 12C

Last part of a query to be evaluated to fetch

only selected rows from the result set:
select *
from emp
order
by sal desc
FETCH FIRST 3 ROWS ONLY;

To select the next set of rows:

select *
from emp
order
by sal desc
OFFSET 3 FETCH NEXT 4 ROWS ONLY;
TOP-N% QUERYING

To query for a percentage of the result set

(rather than an absolute number of rows)
select *
from emp
order
by sal desc
FETCH FIRST 30 PERCENT ROWS ONLY;

And the next batch

select *
from emp
order
by sal desc
OFFSET (0.3*(select count(*) from emp))
ROWS
FETCH NEXT (0.3*(select count(*) from emp))
ROWS ONLY;
BOTTOM-N QUERY IN 12C

Return only the last three rows in the

ordered result set (in the proper order)
select *
from emp
order
by sal desc
OFFSET ((select count(*) from emp)-3)
ROWS
FETCH NEXT 3 ROWS ONLY

or:
select *
from ( select *
from emp
order
by sal asc
FETCH FIRST 3 ROWS ONLY
)
order by sal desc;
 IN-LINE PL/SQL FUNCTIONS AND
PROCEDURES
Procedures are also allowed in-line
In-Line Functions and Procedures can invoke
each other
WITH
procedure increment( operand in out number
, incsize in number)
is
begin
operand:= operand + incsize;
end;
FUNCTION inc(value number) RETURN number IS
l_value number(10):= value;
BEGIN
increment(l_value, 100);
RETURN l_value;
end;
SELECT inc(sal)
from emp
SPECIAL BUSINESS RULE: DEFAULT
VALUE
The default values is the value that should be
inserted for a column when the client has
ignored the column
not provided a value nor indicated NULL
The default value is applied prior to the
execution of the Before Row trigger
So :new.<column_value> has the value that will
be inserted
The Before Row trigger has no built in way to
telling whether the value was provided by the
client or supplied as default by the database
Default value is typically used for auditing
purposes
Note: default values for columns exposed in UI
should be set in the client
COLUMN DEFAULT

Columns can have default values

Static or literals
SQL expressions evaluating to a static
Pseudo-columns like USER and CURRENT_DATE
DO NOT USE SYSDATE! DO NOT USE USER!
References to Application Context parameters
sys_context(USERENV, IP_ADDRESS)..
Some funny value to let the before row trigger know
that the real (complex) default must be calculated

create table citizens

( name varchar2(100) default 'John Doe'
, birthdate date default current_date - 1
, city varchar2(50) default
sys_context('KANE_CTX', 'DEFAULT_CITY' )
, zipcode varchar2(8) default 'XYXYXYXYXQQ'
)
NEW OPTIONS WITH DEFAULT
VALUE
alter table emp
modify (sal number(10,2)
DEFAULT ON NULL 1000
)

alter table emp

modify (empno number(5) NOT NULL
DEFAULT ON NULL EMPNO_SEQ.NEXTVAL
)

create table emp

( empno NUMBER GENERATED BY DEFAULT AS IDENTITY
(START WITH 100 INCREMENT BY 10)
, ...
)
 APPLICATION CONTEXT

Memory area that enables application

developers to define, set, and access
key/value pairs Application
Context
Rapid access in SQL and PL/SQL
Attribute Value Attribute Value
Pairs
Attribute Value

select sys_context('USERENV', 'SESSION_USER')

from dual

l_user:= sys_context('USERENV', 'SESSION_USER')

Two Application Contexts

are always around:
CLIENTCONTEXT and USERENV
APPLICATION CONTEXT
APPEARANCES
Per session (default)
Stored in UGA, just like package state
Globally Accessible (shared across all
sessions)
Stored in SGA
Associated with a Client Identifier
Attributes in a Globally Accessible Application
Context can explicitly be tied to the Client
Identifier
And are only accessible to sessions with that
Client Identifier
TYPICAL WEB ARCHITECTURE
USING CONNECTION POOL

JDBC Connection Pool

Session 1 Session 2 Session 3 Session 4

Package A Package B Package C

globals
PACKAGE STATE IS TIED TO
DATABASE SESSION

JDBC Connection Pool

Session 1 Session 2 Session 3 Session 4

globals

Package A Package B Package C

globals
PACKAGE STATE IS TIED TO
DATABASE SESSION NOT WEB
SESSION

JDBC Connection Pool

Session 1 Session 2 Session 3 Session 4

globals

Package A Package B Package C

globals
APPLICATION CONTEXT TO RETAIN
STATE FOR LIGHT WEIGHT END
USERS

JDBC Connection Pool

Session 1 Session 2 Session 3 Session 4

globals ?

Package A Package B Package C

globals
APPLICATION CONTEXT TO RETAIN
STATE FOR LIGHT WEIGHT END
USERS

JDBC Connection Pool

Session 1 Session 2 Session 3 Session 4

USERENV USERENV

Package A Global Context Package C

globals globals
globals
APPLICATION CONTEXT TO RETAIN
STATE FOR LIGHT WEIGHT END
USERS

JDBC Connection Pool

Session 1 Session 2 Session 3 Session 4

USERENV USERENV USERENV

Package A Global Context Package C

globals globals
globals
PACKAGE GLOBALS: THE STATE OF
THE PACKAGE IN A SESSION
This state is lost when the package is
recompiled
That is undesirable in a highly available
Package
environment
 PACKAGE GLOBALS CAN BE
REPLACED BY APPLICATION CONTEXT
The Application Context is untouched by
recompilation of the package
All globals in the application context retain
Package their values

Application Context
 EBR TO KILL PLANNED DOWNTIME
(BECAUSE OF APPLICATION UPGRADE)

Applicatio
Application X nX
VERSION 1 VERSION
2

ase elease 2
Base Rele R Release 3
TIME TRAVELLING
FLASHBACK

Introduced in 9i
Based on UNDO
Initially only for recovery
As of 11g Total Recall option with
Flashback Data Archive
Controlled history keeping
Look back into history
Query trends (version history)
Difference reporting
Audit trails (Replace journaling tables)
Require trick for transaction history: WHO?
Also: when is the start of history?
OOW 2012 SESSION COMES TO THE
RESCUE
CON8511 - Temporal Database Capabilities
with the Latest Generation of Database
Technology
TOTAL RECALL - FLASHBACK DATA
ARCHIVE IMPROVEMENTS
Complete schema evolution support: all table
definition, partitioning, and space
management DDLs are supported on FDA-
enabled tables.
The metadata information for tracking
transactions including the user context is
now tracked.
This could mean that journaling tables are now
officially deprecated
And the current contents of journaling tables can
even be migrated to Flashback Data Archive
Introduction of SQL 2011 Valid Time Temporal
Modeling
TOTAL RECALL

Import and export of history

Support for import and export using Data Pump
for FDA-enabled tables. Data Pump can now be
used to export and import an FDA-enabled base
table along with its schema-evolution metadata
and historical row versions.
Construct and manipulate the Flashback Data
Archive
import user-generated history
Restore points: Support for the use of named
restore points in AS OF and versions queries
has been added.
Total Recall will (in all likelihood) be part of
every edition of the database including SE
VALID TIME TEMPORAL MODELING

Validity (or effectivity) of facts recorded in a

database is frequently specified through dates or
timestamps
For example begin date and [derived] end date of a
price, membership, allocation, certificate, agreement
This valid time can differ from the transaction time
at which a record is entered into the database
Multiple entries with different, non-overlapping
valid-time periods can exist for a single entity
In 12c the notion of Valid Time is introduced into the
Oracle Database
The valid-time dimension consists of two date-time
columns specified in the table definition (create or alter)
These Valid Time columns specify the period during
which a record is valid
A table can have multiple valid_time markers
CREATING A TABLE WITH VALID
TIME DIMENSION
Table with explicit valid time columns:
CREATE TABLE EMP
( employee_number NUMBER
, salary NUMBER
, department_id NUMBER
, name VARCHAR2(30)
, hiredate TIMESTAMP
, firedate TIMESTAMP
, PERIOD FOR user_time (hiredate, firedate)
);
Table with valid time dimension and implicit
columns:
CREATE TABLE EMP
( employee_number NUMBER
, salary NUMBER
, department_id NUMBER
, name VARCHAR2(30)
, PERIOD FOR contract_time
);

columns valid_time_start and valid_time_end

(TIMESTAMP) are added implicitly
 VALID TIME AWARE FLASHBACK
QUERIES
Select all employees who were employed at a
certain moment in time
SELECT *
FROM EMP AS OF PERIOD FOR user_time
TO_TIMESTAMP('01-JUN-2012 12.00.01 PM')

Perform all queries for records that are valid

at a certain point in time
EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time
( 'ASOF'
, TO_TIMESTAMP('29-JUL-12 12.00.01 PM')
);

Return all records currently (session time)

EXECUTE valid
DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time('CURRENT');

Return all records (default)

EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time('ALL');
 DATABASE IN MODERN
ARCHITECTURE
Mobile WS

Business Tier Cache/Grid

Enterprise (L1, L2, L3)
Service Bus

Services

Standard
Application
Database s
Database Legacy
Application
s
 MULTI TIER ARCHITECTURE

Mobile WS

Business Tier Cache/Grid

Enterprise HTTP REST JDBC (L1, L2, L3)
Service Bus HTTP SOAP JPA (H/EL)
FTP/WEBDAV

Services DB QRCN
HTTP
JMX, JMX
Monitor,
Trace, Audit Stored Encapsulation
Database Procedures Decoupling
Authentication & Caching
Fine Grained SQL Business Logic
Authorization
APPLICATION ARCHITECTURE:
DRIVE APPLICATION FROM META DATA
Agility
Design Time at Run Time
Define part of the application behavior and
appearance through meta-data (outside the
base source code)
The default settings are defined by developers
and deployed along with the application
Read and interpreted at run time
Manipulated and re-read
Application
and re-interpreted at run time
Note: very similar to the way
the database operates:
Data Dictionary is the
meta-data driving the meta
behavior of the database
SEPARATE
BASE DATA AND CUSTOMIZED DATA
If a value is changed during site-level
implementation
Or run time customization
It should be kept apart from the base meta-
data
To prevent overwriting customized data when
the new release arrives
To allow for (temporarily) reverting to base data
A simple solution: the Complex View with two
underlying tables approach
Note: Select ORIGINAL_NAME
For Update Of IO trg
is not allowed Customized
Values
New release BaseValues
REPLACE THE ORIGINAL SINGLE TABLE
WITH A TWO-TABLE BASE/CUSTOM SPLIT
rename <original> to <base>
create table <customizations>
as
select * from base where rownum = 0
create or replace view <original>
as
select * from <customizations>
union all
select * from <base> b
left outer join
<customizations> c
on (b.id = c.id)
where c.rowid is null
REPLACE THE ORIGINAL SINGLE TABLE WITH A
TWO-TABLE BASE/CUSTOM SPLIT (2)

create or replace trigger handle_insert_trg

instead of insert on original
for each row
begin
insert into <customizations> (col, col2,)
values(:new.col, :new.col2,);
end;

create or replace trigger handle_update_trg

instead of update on original
for each row
begin
update <customizations>
set col = :new.col,
where id = :new.id ;
if sql%rowcount = 0
then
insert into <customizations> (id, col, col2,)
(select id, :new.col, :new.col2 frombase where id = :new.id);
end if;
end;
 VERY SIMILAR TO THE ARCHITECTURE
OF PLUGGABLE DATABASES

New release of
Oracle Database ROOT

PDB
APPLICATION ARCHITECTURE: NO
SQL
NO SQL
Complex SQL is
hidden away inside
the database
Cache to not have Web Browser
to query all the time
from the database
and to not take
the overhead of a
commit for not so
JEE Application Server
important data NO SQL
Process first in
memory, on
middle tier
(BigData and CEP) -
and only persist
what is useful
RDBMS
SQL
QUERY RESULT CHANGE
NOTIFICATION
Continuous Query Notification:
Send an event when the result set for a query
changes
Background process calls PL/SQL Handler or Java
Java
Listener or OCI client when the Listener
commit has occurred
Event contains rowid
of changed rows
Used for:
Refreshing specific
data caches (middle
tier, global context)
(custom) Replication
PL/SQL
CONTINUOUS PROCESSING OF DATA
STREAMS USING CQL
Aggregation, Spot deviation, Match on
complex patterns
WHO IS AFRAID OF RED, YELLOW
AND BLUE

Table Events
Column Seq number(5)
Column Payload varchar2(200)
 SOLUTION USING LEAD

With LEAD it is easy to compare a row with

its successor(s)
As long as the pattern is fixed, LEAD will suffice
with look_ahead_events as
( SELECT e.*
, lead(payload) over (order by seq) next_color
, lead(payload,2) over (order by seq) second_next_color
FROM events e
)
select seq
from look_ahead_events
where payload ='red'
and next_color ='yellow'
and second_next_color='blue'
FIND THE PATTERN RED, YELLOW
AND BLUE
Using the new 12c Match Recognize operator
for finding patterns in relational data
SELECT *
FROM events
MATCH_RECOGNIZE
(
ORDER BY seq
MEASURES RED.seq AS redseq
, MATCH_NUMBER() AS match_num
ALL ROWS PER MATCH
PATTERN (RED YELLOW BLUE)
DEFINE
RED AS RED.payload ='red',
YELLOW AS YELLOW.payload ='yellow',
BLUE AS BLUE.payload ='blue'
) MR
ORDER
BY MR.redseq
, MR.seq;
MATCH_RECOGNIZE FOR FINDING
PATTERNS IN RELATIONAL DATA
The expression MATCH_RECOGNIZE provides
native SQL support to find patterns in sequences
of rows
Match_
Table
Recognize Select &
Source & Process Order By
Where
and Filter

Match_recognize returns Measures for selected

(pattern matched) rows
Similar to MODEL clause
Match Conditions are expressed in columns from
the Table Source, aggregate functions and
pattern functions FIRST, PREV, NEXT, LAST
Patterns are regular expressions using match
conditions to express a special sequence of rows
satisfying the conditions
 DID WE EVER CONSECUTIVELY HIRE
THREE EMPLOYEES IN THE SAME JOB?
Find a string of three subsequent hires where
each hire has the same job
Order by hiredate, pattern is two records that
each have the same job as their predecessor

SELECT *
FROM EMP
MATCH_RECOGNIZE
(
ORDER BY hiredate
MEASURES SAME_JOB.hiredate AS hireday
, MATCH_NUMBER() AS match_num
ALL ROWS PER MATCH
PATTERN (SAME_JOB{3})
DEFINE
SAME_JOB AS SAME_JOB.job = FIRST(SAME_JOB.job)
) MR
THE SHOPPING ALGORITHM
THE SHOPPING ALGORITHM

shopForItem Item ( String itemName) {

driveToShop;
Item item = buyItemAtShop ( itemName);
driveHomeFromShop;
return item;
}
GET THIS WEEKS GROCERIES

getGroceries Item[] ( String[] shoppingList) {

Item[] items = new Item[ shoppingList.length];

for (int i=0; i < shoppingList.length; i++) {

items[i] = shopForItem (shoppingList[i]);
}
return items;
}
PENSION FUND SEPTEMBER 2012

Employer < >

Participants

Job & Benefits

FETCHING THE DATA OF THE PENSION
FUND FOR THE WEB APPLICATION

1 record
< > select *
from employers
where id = < 324>

select * 100s records

from participants
where employer_id = < 324>

select * 10s records

from benefits
where participant_id = <#>
REPORTING ON MANY EMPLOYERS

100s records
select *
from employers 1 query

select * 10k records

from participants 100s queries
where employer_id = <#>

select * 100k records

from benefits 10k queries
where participant_id = <#>
APPLICATION ARCHITECTURE
BULK RETRIEVE
Have the database bulk up the data retrieval
Return Ref Cursor, Types and Collections or
JSON/XML

Benefits Package

select *
from employers
where id in <some set> select *
from participants
where employer_id in <some set>
select b.*
from benefits b join participants p
on (p.id = b.participant_id)
where p.employer_id in <some set>
 APPLICATION ARCHITECTURE
SERVICE ENABLING
WebLogic Server Database

Re
OO
Native DB
WebService

lat
HTTP EPG
Java/JEE

XM on
PL/SQL
package

a
SOA Suite JDBC
AQ

L
View
Oracle Other Table
Service Bus (Email, FTP/File, XML
DB
XMPP/Chat)

Chat/IM XMPP
Server File/FTP Server
Email Server
 XML/JSON
Relational/Oracle Type

JEE Server Database

11g Native DB
WebService
HTTP
10g
http
ADF BC
EPG
JSON/ CSV
Java App SDO /SDO WS XML PL/SQL
XML & XSD
WS
JAX-WS
package
Ref Cursor
JPublisher
DB Types & Coll
SOA WS JDBC XML
8i Types
Suite AQ AQ
Adapters

JMS Queue JMS

utl_file, View
JMS
BFILE,
Oracle EJB
EJB/JPA URITYPE
Service Pojo Other Table
(Email, FTP/File, 9i XML
Bus File
DB
FTP XMPP/Chat)
UMS

Chat/IM XMPP
Server
File/FTP Server
Email Server
BUSINESS RULES

Data Oriented Rules or Data Constraints

Declarative support in database
For referential integrity
Order must be for a Customer
For attribute and tuple rules
Salary must be numeric,
Hiredate may not be in the future,
End date must come after begin date
No declarative support for complex data rules
across multiple records and tables
A department in France may not have less then
20% female employees
Order items of type weapon may not be part of
an order that ships around Christmas
BUSINESS RULES
WHERE AND HOW TO IMPLEMENT
Criteria:
Safe
Well performant
Reusable and maintainable
Productive to implement
Options
Client side
JavaScript
Middle-tier
Java, Enterprise Service Bus
Database
Constraints and triggers are statement level i/o
transaction level
 RDBMS NOT ALWAYS EXCLUSIVELY
ACCESSED THROUGH ONE LAYER

SOA, ESB,
WebServices

Batch Bulk
Processes Standard
Applications
Database

Data Replication & Legacy

Synchronization Applications
 11G VIRTUAL COLUMNS

Add columns to a table based on an

expression
Using real columns, SQL Function and User
Defined Functions
No data is stored for Virtual
Columns, only meta-data
VIRTUAL
Virtual Columns can be
indexed

alter table emp

ADD
(income AS (sal +
nvl(comm,0)))
 UNIQUENESS RULES
USING VIRTUAL COLUMNS
Business Rule:
Not more than one manager per department

alter table emp

add constraint only_one_mgr_in_dept_uk
unique (one_mgr_flag)

alter table emp

ADD
( one_mgr_flag as
( case when job ='MANAGER'
then deptno
end
)
)
 CHALLENGE: ORDERS BELONG TO A
CUSTOMER IN ONE OF TWO TABLES
The Orders table contains Order records for
customers either Dutch or Australian
customers
These customers are stored in two different
tables
Can we implement referential integrity to
ensure that the orders customer exists?

OZ_CUSTOMER

?
ORDER
Id
Name
Country DUTCH_CUSTOMER
Customer_Id
. Id
Name
 USING VIRTUAL COLUMNS
IN FOREIGN KEY RELATIONS
A foreign key can be created on a Virtual
Column
That means for example we can have a single
column with some id
And two virtual columns with CASE expressions
that produce NULL or the ID value
With Foreign Keys on the Virtual Columns

OZ_CUSTOMER
ORDER
Id
Country Name
DUTCH_CUSTOMER
Customer_Id
Dutch_id (VC) Id
Australian_id (VC) Name
 USING VIRTUAL COLUMNS
IN FOREIGN KEY RELATIONS

alter table orders alter table orders

add (australian_ctr_id as add constraint odr_ocr_fk
(case country foreign key (australian_ctr_id)
when 'OZ' references oz_customer (id)
then customer_id
end))
OZ_CUSTOMER
ORDER
Id
Country Name
DUTCH_CUSTOMER
Customer_Id
Dutch_id (VC) Id
Australian_id (VC) Name
alter table orders
add (dutch_ctr_id as alter table orders
(case country add constraint odr_dcr_fk
when 'NL' foreign key (dutch_ctr_id)
then customer_id references dutch_customer
end)) (id)
FOREIGN KEY SHOULD ONLY REFER
TO CERTAIN RECORDS USING VC
Foreign Key can reference a UK based on a
Virtual Column
That allows a conditional foreign key or a
foreign key that can only reference specific
records in the referenced table
Only refer to Women in the PEOPLE table for the
Mother Foreign Key
Only refer to Values in the Domain Values table
for the Domain Name == COLORS
 RESTRICTED FOREIGN KEYS USING
VIRTUAL COLUMNS
alter table domain_values
add (country_value
alter as
table domain_values
(case domain_name
add (country_value as
alter table domain_values whendomain_name
(case 'COUNTRIES'
then'COUNTRIES'
when domain_value
add (color_value as end))
then domain_value
(case domain_name end))
when 'COLORS'
then domain_value DOMAIN_VALUES
end))
CARS Id
Domain_Name
Domain_Value
ID
Color_Value
Make Gender_Value
Type OrderStatus_Value
Color Country_Value
Year ShipmentMethod_Value

alter table cars

add constraint car_clr_fk
foreign key (color)
references domain_values
(color_value)
LACK OF WATERTIGHTNESS
IN TRIGGER BASED RULE
VALIDATOIN
 VALIDATION
Statement time validation means:

DML in different session

DML More DML Commit

validation validation
To prevent leakage we should validate at commit time
Logically correct as transaction is the logical unit
Effects from other sessions between statement and
commit are taken into account
However: Oracle unfortunately does not provide us
with a pre-commit or on-commit trigger
Workarounds:
Dummy Table with Materialized View On Commit Refresh
and Trigger on Materialized View
Do a soft-commit by calling a package to do the actual
commit that will first do transaction level checks
Supported by a deferred check constraint that is violated by
each operation that potentially violates a business rule
 SAFE SOLUTION: USE CUSTOM
LOCKS
Prior to validating a certain business rule for
a specific record acquire a custom lock
That identifies both Rule and Record
Using dbms_lock

DML in different session

DML More DML Commit

validation validation

When a record is being validated for a certain

rule, other sessions have to wait
The commit (or rollback) releases all locks
Validation in a different session will include
all committed data
SUMMARY

Inline Views
Defaulting
Application Context
Flashback and the time dimension
NoSQL means smart SQL
Cache refresh driven by change notification
Streaming analysis before persisting
Decoupling galore
Bulk retrieval
Service enabling
Business Rules
EBR
12c promises even more