Aligning the Warehouse and the Web
We now examine the M-DE component operation
in detail. Logically we divide the model into four
components. In the diagram these are labeled Filter,
Modify, Analyze and Format respectively.
Firstly, the Filter component takes a query from a
user and checks that it is valid and suitable for further
action. In some cases the user is directed immediately
to existing results, or may request a manual override.
The filter may be customized to the needs of the user(s).
Next the Modify component handles the task of query
modification. This is done to address the uniqueness
of search criteria present across individual search en-
gines. The effect of the modifications is to maximize
the success rates of searches across the suite of search
engines interrogated.
The modified queries are sent to the search engines
and the returned results are analyzed, by the Analyze
component, to determine structure, content type
and viability. At this stage redundant documents are
eliminated and common web file types are handled
including .HTML, .doc, .pdf, .xml and .ps. Current
search engines sometimes produce large volumes of
irrelevant results. To tackle this problem we must con-
sider semantic issues, as well as structural and syntactic
ones. Standard IR techniques are applied to focus on
the issue of relevance in the retrieved documents col-
lection. Many tools exist to aid us in applying both IR
and data retrieval techniques to the results obtained
Figure 2. Meta-data engine operation
Handle Query Filter
from D.W.
Meta-Data
format
Provide Results to
D.W.
from the web (Manning et al, 2007; Zhenyu et al, 2002;
Daconta et al, 2003). A
Semantic Issues
In the Data Warehouse much attention is paid to retaining
the purity, consistency and integrity of data originating
from operational databases. These databases take sev-
eral steps to codify meaning through the use of careful
design, data entry procedures, database triggers, etc.
The use of explicit meta-data safeguards the meaning
of data in these databases. Unfortunately on the web
the situation is often chaotic. One promising avenue in
addressing the issue of relevance in a heterogeneous
environment is the use of formal, knowledge representa-
tion constructs known as Ontologies. These constructs
have again recently been the subject of revived interest
in view of Semantic Web initiatives. In our model we
plan to use a domain-specific ontology or taxonomy
in the format module to match the results’ terms and
hence distinguish relevant from non-relevant results
(Ding et al, 2007; Decker et al, 2000; Chakrabarti, 2002;
Kalfoglou et al, 2004; Hassell et al, 2006; Holzinger
et al, 2006).
Modify
Submit Modified
Queries to S.E.
Analyse
Handle Results
from S..E.