Rajan Dhabalia LSP Research Paper

Name: Rajan Dhabalia
LSP : Logic Score Preference

Table of Contain :
(1) Introduction……………………………………………………………………………………………………………………………………………….. 3
(2) Goal of the project……………………………………………………………………………………………………………………………………….4
(3) Project requirement…………………………………………………………………………………………………………………………………….4
User Profile
Tools used
(4) Steps for design quality evaluation………………………………………………………………………………………………………………..6
(5) Identify performance attribute………………………………………………………………………………………………………………………7
(6) Elementary criteria……………………………………………………………………………………………………………………………………….10
(7) Aggregation of preference………………………………………………………………………………………………………………………….…28
(8) Competitive system…………………………………………………………………………………………………………………………………….…34
(9) Result…………………………………………………………………………………………………………………………………………………………....38
(1) EVALUATION REPORT FOR THE SearchEngine PROJECT [SearchEngine.txt] ……………………………………………..38
(2) DETAILED EVALUATION RESULTS FOR THE SearchEngine PROJECT [SearchEngine.lst]……………………………….44
(3) Result of SearchEngines [SearchEngine.res]……………………………………………………………………………………………...70
(4) SUMMARY OF RESULTS FOR THE SearchEngine PROJECT [SearchEngine.sum]…………………………………………..72
1. Introduction
The software Quality Analysis is a measure of properties of a piece of software or its
specifications. The direct measurement of software quality is quite difficult due to lack of
quality factor measurement. To resolve this measurement problem, there is a model which
measures the quality of the software in terms of the attributes, specifications and
characteristics. This model is known as LSP (Logic Score Preference) .When client gives
specifications of the software to the developer then client expects the good quality of
software from developers. Hence, to decide the quality of software we can use this LSP
model.
This model validates following software quality attributes.
(1) Functionality
Suitability
Accuracy
Security
Interoperability
Compliance
(2) Usability
Understandability
Learn ability
Operability
(3) Performance
Processing time
Throughput
Resource consumption
(4) Maintainability
(5) Portability
(6) Reusability
In LSP, the features are decomposed into above aggregation blocks. And this decomposition
continues with in the each block until the all the lowest level features are directly measurable
and makes tree of decomposed features. And for each feature, an elementary criterion is
defined. And LSP calculates elementary preference for each criterion and then aggregate all
of them to calculate final global preference. And this global preference shows the quality of
the software. We can calculate global preference for different systems and we can analyze
and compare the systems’ quality.
2. Goal of the Project
The very first tool used for searching on the internet was Archie in 1990. And from that day
the era of search engines begins. And effective search engines such as W2Catalog and Aliweb
were introduced in 1993. After that, search engines were improved by adding complexity and
functionality. And now days we have efficient and qualitative search engines such as Google,
Yahoo Search, bing, Ask, etc. Hence, the goal of this project is to analyze the functionality
and the complexity of those search engines. And we will find how these search engines satisfy
the user requirements.
But before the evaluation, evaluator must know the user requirements. It means evaluator
must know that who is the user and what user expects from the system. And next stage is that
evaluator must have the expert knowledge in the system. Hence, evaluator can sort out the
software attributes very well. And for further stage, evaluator should select the system
components which can be compared between two systems.
Hence, as being an evaluator we will choose the different search engines to compare their
system quality. And we will choose different attributes for the search engine system. And
finally we will give different scale of preference to each attribute for every search-engine
(Google, Yahoo Search, bing, Ask). And after going through whole process of LSP, we will find
out global preference of each search-engine and analyze their quality.
In this project we will put more focus on building a comprehensive evaluation model for five
search engines: Google, Yahoo Search, bing, Ask, altavista. This model aggregates all the
features those reflect the functionality and usability of the search engines and generates a
compound indication for the overall quality. Hence finally it reflects the measurement of user
satisfaction for all aspects of the search-engines.
3. Project Requirements
Here we discuss initial requirements to evaluate this model.
3.1 User Profile

To evaluate Logic score preference model for different search-engine, we need profile
of user that uses these search engine. And also we need to have purpose of user to use search-
engine.
User: Student
Purpose:
Search study related documentations
Research papers
Presentation slides [pdf, ppt, etc. files]
Search books on web
Search images, videos and audio related to projects of study.
Search geographical location of universities or any other places
3.2 Tools Used

LSPcalc128
ANSY
Google Chrome 4.1
Internet Explorer 7.0
Secure Shell Client
4. STEPS FOR DESIGN QUALITY EVALUATION
Step 1:
Develop a hierarchical model for quality characteristics and attributes (i.e. A1 …. An):
Such as so with the help of them, we can compare the systems. This step defines and specifies
the quality characteristics and attributes, grouping them into a model. For each quantifiable
attribute Ai, model associates a variable Wi, which is a weight of that attribute in the system
and it can take a real value.
- So, for search engine I have developed attribute-tree [4.1]
Step 2:
Define criterion function for each attribute, and apply attribute measurement for each
search-engine.
Elementary evaluation criteria specify how to measure quantifiable attributes. The result is an
elementary preference, which can be interpreted as a Degree of satisfied requirement. For
each attribute, it is necessary to establish an acceptable range of values and define a
function, called the elementary criterion. This function maps the measured value in the
numerical domain.
- So, for search-engine I have developed elementary preference and attribute measurement
values in section [5.1]
Step 3:
- Evaluating elementary preferences
- Logic aggregation of preferences
- So, for search engine I have developed logic aggregation preference using pictorial diagram.
Aggregators are chosen based on the user needs which are expressed as relationship between
inputs to be simultaneity, replaceability and neutrality. There are some inputs which are to be
satisfied simultaneously and finally we aggregate the preferences and compute the overall
suitability of the system.
Step 4:
- Evaluating competitive functions
- Here, we have five different such engines such as Google, Yahoo Search, bing, Ask, altavista.
And we will evaluate competitive feature analysis for each above search-engines.
Step 5:
- Ranking and selection of the best system and analysis of the system
- And finally using LSPCalc, we can give rank to each search-engine according to their global
reference. And select best among them.
- How to use LSPCalc128 tool ?
Ans: Create .cri and .dat files
Create a directory in the projects folder named with your project
Create two files as projectname.cri and projectname.dat
By default the LSPcalc128 looks into projects folder to search for all the projects
And then we can open any client shell to access the LSPcalc128
Run the executable and follow the instructions on the screen of this tool.
5. Identifying performance attributes
5.1: System attribute tree
Attribute tree
1.1 Searching Input

1.1.1 Text Searching
1.1.1.1 Keyword Searching
1.1.1.1.1 Search with one keyword
1.1.1.1.2 Search with group of keywords
1.1.1.2 Statement Searching
1.1.1.2.1 Search exact statement
1.1.1.2.2 Search part of statement
1.1.1.2.3 Case sensitive Search
1.1.1.2.4 Ignore stop words (the,a,an,..etc)
1.1.1.3 Search with different language
1.1.1.4 Numeric Expression Search
1.1.1.4.1 Search numeric expression in documents
1.1.1.4.2 Compute numeric expression
1.1.1.5 Relational Searching
1.1.1.5.1 Abbreviation Search
1.1.1.5.2 Synonyms Search
1.1.1.5.3 Stemming Search
1.1.1.5.4 Misspell Corrected Search
1.1.1.6 Operator Searching
1.1.1.6.1 Including Search (using ‘+’ operator)
1.1.1.6.2 Excluding Search (using ‘-‘ operator)
1.1.1.6.3 Combinational Search (using ‘*’ operator)
1.1.1.6.4 Optional Search (using ‘OR’ operator)
1.1.2 Multimedia Search
1.1.2.1 Image Searching
1.1.2.1.1 Filename search
1.1.2.1.2 Image link search
1.1.2.1.3 Adjacent text search
1.1.2.2 Video Searching
1.1.2.2.1 Popularity based Search
1.1.2.2.2 Content based Search
1.1.2.2.3 Content Controlled Search
1.1.2.3 Audio Searching
1.2 Searching filter

1.2.1 Security filter
1.2.1.1 Pages with Spam, doorway
1.2.1.2 Duplicate Content
1.2.2 Citation filter
1.2.2.1 Adult content filter
1.2.2.2 Casino content filter
1.2.3 Domain filter
1.2.3.1 Pages related to same content
1.2.3.2 Pages related to same website
1.2.3.3 Linked page
1.2.4 Extended filter
1.2.4.1 RSS support pages
1.2.4.2 Usage rights
1.2.4.3 Numeric range filter
1.2.4.4 field to search keyword [title,text,URL,Link]
1.2.5 File specific filter [pdf, word, excel sheet,..]
1.2.6 Broken link filter
1.2.7 Time filter
1.2.7.1 Page created time
1.2.7.2 Recent update time
1.2.8 Location filter
1.2.8.1 Location of searching user [i.e. weather forecast]
1.2.8.2 Location of country
1.3 Specific activity Search

1.3.1 Weather
1.3.2 Blog
1.3.3 Movie time
1.3.4 Sport score
1.3.5 Stock price
1.3.6 Literature [books,..]
1.3.7 Maps
2. Usability
2.1 Interface usability
2.1.1 Interface visibility
2.1.2 Operability
2.1.3 Customization
2.1.3.1 Customization of page size
2.1.3.2 Customization of page rank
2.2 Result Evaluation
2.2.1 Result visibility
2.2.2 Accessibility of results
2.2.3 Availability of cached result
2.3 User guide
2.3.1 Online help
2.3.2 Manual user-guide
2.3.3 FAQ
2.3.4 Related tutorial material
3. Performance
3.1 Loading time
3.1.1 Load page time
3.1.2 Automatic search suggestion time
3.1.3 Result evaluation
3.1.3.1 Time to evaluate best result
3.1.3.2 Time to evaluate top N result
3.2 Resource Consumption
4. Reliability
4.1 User satisfaction
4.1.1 Popular pages results
4.1.2 High rank pages results
4.1.3 Coverage of user need
4.2 Confusion matrix
4.2.1 Accuracy
4.2.2 Precision
4.2.3 Recall
4.2.4 Specificity
6 : Elementary Criteria
Point additive discrete absolute criteria
[6.1] 1.1.1 Text Searching

Keyword Searching Weight
Search with one - Search engine must search text with the single keyword. And that
keyword single word may relate to any issue. It may from any study subjects
(Computer, Mathematics, Mechanical, etc..), any dictionary word, or 50
any un-traditional word.
- So, search engine should have capability to cover N no of words.
- Efficiency for Dictionary words - 5
- Efficiency for Non-Dictionary words - 5
Search with one keyword [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Search with group of - Search engine should have capability to correlate group of keywords 50
keywords and find combination of these keywords on the web.
- Find all words of group – 8
- Find some of words of group - 2
Group of keywords [Rank of Accuracy]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Statement Searching
Search exact - Search REL= 100 * S/ S max 40
statement Efficiency to search exact expression on web as user wants
Search exact statement [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Search parts of - Search REL= 100 * S/ S max 60

statement Efficiency to search all divided parts of the expression on web.
Search part of statement [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Case sensitive - if search-engine is case sensitive - 0 20
Search - otherwise - 1
Case sensitive Search [Yes=1 and No=0]
0 1
0 10 20 30 40 50 60 70 80 90 100
Ignore stop words - Ignore REL= 100 * Ignore/ Ignore max 80

(the,a,an,..etc) Efficiency to search by removing remove words for efficient search
Ignore stop words [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Search with different - Number of languages which search engine can support. 20
language
Search with different languages[No of languages]
1 25 51
0 10 20 30 40 50 60 70 80 90 100
Numeric Expression Search

Search numeric - Search REL= 100 * S/ S max 80
expression in Efficiency to search Numeric expression
documents i.e. population of country
Numeric exp in document [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Compute numeric - Number of mathematical expression search engine can compute. 20

expression - i.e. Mathematical operation +,-,*,/,sinX,tanX,sqrt,etc..
Compute numeric Exp [No of Expression]
4 15 30
Relational Searching
Abbreviation Search - AS REL= 100 * AS/ AS max 20
Efficiency to search Abbreviation on web as user wants
Abbreviation Search [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Synonyms Search - SS REL= 100 * SS/ SS max 30

Efficiency to search synonyms of searching word
Synonyms Search [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Stemming Search - SS REL= 100 * SS/ SS max 20

Efficiency to search with stemming technique for efficient search.
Stemming Search [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Misspell Corrected - MS REL= 100 * MS/ MS max 30

Search Efficiency to search with correction of misspell word
Misspell Corrected Search [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Operator Searching
Including Search - IS REL= 100 * IS/ IS max
(using ‘+’ operator) Efficiency to search multiple expression with the operator ‘+’
55
Including Search [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Excluding Search - ES REL= 100 * ES/ ES max

(using ‘-‘ operator) Efficiency to search with subtraction facilities , so user can search
specific part from whole part of expression
10
Excluding Search [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Combinational - CS REL= 100 * CS/ CS max

Search (using ‘*’ Efficiency to search combinational search-expression on web as user
operator) wants
15
Combination Seach [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Optional Search - OS REL= 100 * OS/ OS max

(using ‘OR’ Efficiency to search with “Optional search” facility on web as user
operator) wants 20
Optional Search [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Image Searching
Filename search - FNS REL= 100 * FNS/ FNS max
Efficiency to search image as its file name on the web
20
Filename Search [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Image link search - ILS REL= 100 * ILS/ ILS max
Efficiency to search Image on web as user wants
30
Image link Search [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Adjacent text search - ATS REL= 100 * ATS/ ATS max 50

Efficiency to search image using adjacent text with target image on the
web.
Adjacent text Search [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Video Searching
Popularity based - VPS REL= 100 * VPS/ VPS max
Search Efficiency to search video as its popularity 50
Popularity based Search [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Content based - VCS REL= 100 * VAS/ VAS max

Search Efficiency to search video using content related to that video.
Content based Search [Level of efficiency] 50

10 5 1
0 10 20 30 40 50 60 70 80 90 100
Content Controlled - VCCS REL= 100 * VCCS/ VCCS max 25

Search Efficiency to search video with some specific user defined criteria.
- Funny, drama, Action, etc..
Content Controlled Search [Level of efficiency]
10 5 1
Audio Searching - AudioS REL= 100 * AudioS/AudioS max
Efficiency to Search audio as per user requirement.
Audio Searching [Level of efficiency] 30

10 5 1
0 10 20 30 40 50 60 70 80 90 100
[6.2] 1.2 Searching Filter
Security filter Weight

Pages with Spam, - SSS REL= 100 * SSS/ SSS max
doorway Efficiency of Security Spam filter is to prevent search of web-sites 55
which cause to download spam and doorway in user system.
Pages with Spam,doorway [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Duplicate Content - DCS REL= 100 * DCS/ DCS max 45

Efficiency to prevent searching duplicate content in fraud web-
sites to follow copyright right rule.
Duplicate Content [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Citation filter
Adult content filter - ACS REL= 100 * ACS/ ACS max
Efficiency of filter to prevent Searching Adult content on the web.
Adult Content filter [Level of efficiency] 90

10 5 1
0 10 20 30 40 50 60 70 80 90 100
Casino content filter - CCS REL= 100 * CCS/ CCS max 10
Efficiency of filter to prevent Searching Casino and gambling
content on the web.
Casino content filter [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Domain filter
Pages related to same - DPS REL= 100 * DPS/ DPS max
content Efficiency of filter to search pages which has same kind of 45
contents.
Content Related Pages [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Pages related to same - DWS REL= 100 * DWS/ DWS max 35

website Efficiency of filter to pages which are related to same website
domain.
Website related website [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Linked page - DLS REL= 100 * DLS/ DLS max 20

Efficiency of filter to search pages which are linked with each-
other.
Linked Page [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Extended filter
RSS support pages - RSS REL= 100 * RSS/ RSS max 35
Efficiency of filter to search RSS support pages.
RSS support pages [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Usage rights - US REL= 100 * US/ US max

Efficiency of filter to give access to user who has usage right for
target searched page. 30
Usage rights [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Numeric range filter - NS REL= 100 * NS/ NS max 35

Efficiency of filter to search with sorting facility [For numeric
range sort]
Numeric range filter [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
field to search keyword Search with specific field

[title,text,URL,Link] - URL of page 55
- Title
- Specific page text
- Link of the page
Field to search keyword [No of Field]
1 2 4
0 10 20 30 40 50 60 70 80 90 100
File specific filter [pdf, - Ability to find out different kind of file on the web.
word, excel sheet,..] - i.e. .ppt, .doc, .pdf, etc..
35
File specific filter [No of types]
1 15 30
0 10 20 30 40 50 60 70 80 90 100
Broken link filter - BLS REL= 100 * BLS/ BLS max

Efficiency of filter to prevent to search broken link pages or pages
no longer exist.
15
Broken link filter [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Time filter
Page created time - PTS REL= 100 * PTS/ PTS max 20
Efficiency of filter to search page with created year or time for
that page.
Page created time [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Recent update time - RUS REL= 100 * RUS/ RUS max

Efficiency of filter to search page with recent update result.
Recent update time [Level of efficiency] 80

10 5 1
0 10 20 30 40 50 60 70 80 90 100
Location filter
Location of searching user - LS REL= 100 * LS/ LS max
[i.e. weather forecast] Efficiency of filter to search pages related to geographic location
of user. 30
Location of user [Level of efficiency]

10 5 1
Location of country - LCS REL= 100 * LCS/ LCS max 70

Efficiency of filter to search pages related to country to which
user belongs.
Location of Country [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
[6.3] 1.3 Specific activity Search
Weather - WS REL= 100 * WS/ WS max

Efficiency to search efficient weather result.
8
Wether [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Blog - BS REL= 100 * BS/ BS max 15

Efficiency to search Blogs on web as user wants
Blog [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Movie time - MS REL= 100 * MS/ MS max

Efficiency to search Movie-timing of cinemas on web as user
wants 8
Movie time [Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Sport score - SSS REL= 100 *SSS/ SSS max 20

Efficiency to search latest different sports score on web as user
wants
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Stock price - SPS REL= 100 * SPS/ SPS max

Efficiency to search latest stock-price on web as user wants
14
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Literature Efficiency to search different field related books and literatures on 15

[books,..] web as user wants
author – 3 points
content – 4 points
publisher – 3 points
1 5 10
0 10 20 30 40 50 60 70 80 90 100
Maps Map scoring points 20

Public transportation trace root – 2
Driving direction – 2.5
Walking root – 0.5
Save Map – 0.5
Write Map description – 0.5
Review of place - 1
Search place by place-properties - 1
Printable Map – 1
Maps for mobile or handy-instruments - 1
1 5 10
0 10 20 30 40 50 60 70 80 90 100
[6.4] 2. Usability
Interface usability Weight

Interface visibility - Specification of visibility
Textbox to search – 3 40
Target Search Button – 3
Select search options [image,video] – 1
Without redundant component – 1
Easy and Clear - 2
Interface visibility [Level of visibility]
1 5 10
0 10 20 30 40 50 60 70 80 90 100
Operability - Specification of visibility 60

No of Input component – 4
No of click to search – 3
Simple Searching criteria – 3
Operability [Level of user operability]
1 5 10
0 10 20 30 40 50 60 70 80 90 100
Customization
Customization of page - Facility to customize page according to user need
size No of Results per page – 8 20
Searching customization – 1.5
Page-Theme selection – 0.5
Page size[Level of customization]
1 5 10
0 10 20 30 40 50 60 70 80 90 100
Customization of page - CPR REL= 100 *CPR/ CPR max 80

rank Efficiency to give importance of user’s taste[Using users’ visits on
the pages] and give high page rank count those pages and show
those pages on top results.
Page Rank [Level of page rank algorithm]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Result Evaluation
Result visibility - Result visibility
Top N result on the page – 3
Easy to access – 5 20
Result without redundant data on page – 2
Result visibility [No of result per page]
1 5 10
0 10 20 30 40 50 60 70 80 90 100
Accessibility of results - Accessibility 80

No of click – 4
Brief info of result on result page – 3
Result according to ranking – 3
Accessibility of Results [Level of accessibility]
1 5 10
0 10 20 30 40 50 60 70 80 90 100
Availability of cached - CR REL= 100 *CR/ CR max

result Efficiency to store and use cache result for next result evaluation. 20
cached Result[Level of efficiency]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
User guide
Online help - ONH REL= 100 *ONH/ ONH max
Availability of online help to guide users.
20
Online Help [Level of satisfied Material]
10 5 1
Manual user-guide - UG REL= 100 *UG/ UG max
Availability of manual user-guide to guide users. 30
Manual user-guide [Level of Quality of Guide ]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
FAQ - Frequent ask question

No of FAQ provided by the search engine to solve user doubts. 20
Frequent Ask Questions [No of Questions]
5 25 45
0 10 20 30 40 50 60 70 80 90 100
Related tutorial - TM REL= 100 *TM/ TM max

material Availability of Related tutorial materials to guide users.
30
Tutorial Material [Rank of Material Quality ]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
`
[6.5] 3. Performance
Loading time Weight

Load page time - Time taken by page[Search-engine] to load on user system 70
[in Second]
Load page time [seconds]
12 8 4
0 10 20 30 40 50 60 70 80 90 100
Automatic search - Time taken by suggestion-search to assist user for searching

suggestion time [in Second]
30
Suggestion Time [Seconds]
3 1 0.5
0 10 20 30 40 50 60 70 80 90 100
Result evaluation
Time to evaluate best - Time to evaluate result for target search. 80
result [in Second]
Time to having result [Seconds]
18 11 4
0 10 20 30 40 50 60 70 80 90 100
Time to evaluate top N - Time taken to evaluate top N result [in Second]
result 20
Time to having top N Results [ Seconds]
16 8 0.9
0 10 20 30 40 50 60 70 80 90 100
Resource - Memory occupied by the HTML source page of the search-

Consumption engine.
[Kilo-bytes] 30
Resource Consumption [Page Size [kb]]
520 20
0 10 20 30 40 50 60 70 80 90 100
[6.6] 3. Reliability
User satisfaction
Popular pages results - Efficiency for reliable search
Popular pages - 4 30
Informative pages - 3
Trustworthy pages - 1
Ability to search target page of specific web-site
domain rather Home Page – 2
Popular Pages Results [Level of reliability]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
High rank pages - Efficiency to computing page rank and evaluate result using 20
results ranking
Rank pages results [Level of reliability]
10 5 1
0 10 20 30 40 50 60 70 80 90 100
Coverage of user need - Efficiency to satisfy and cover need of user
Coverage need [Level of user satisfaction] 50

10 5 1
0 10 20 30 40 50 60 70 80 90 100
Confusion matrix for result

Accuracy - Accuracy of the Result 20
Accuracy [% accuracy]
100 50 1
0 10 20 30 40 50 60 70 80 90 100
Precision - Precision for evaluated result for a target search

Precision [% precision]
100 50 1
30
0 10 20 30 40 50 60 70 80 90 100
Recall - Recall for evaluated result for a target search
Recall [% recall] 20
100 50 1
0 10 20 30 40 50 60 70 80 90 100
Specificity - Specification for evaluated result for a target search 30
Specificity [% Specificity]
100 50 1
0 10 20 30 40 50 60 70 80 90 100
7. Ranking and Aggregation of the preferences
The system requirement tree has 73 performance variables as listed above. We make a weighted graph of
the performance variables and using aggregators .
7.1Functionality
7.1.1 Text Searching :
[Fig: 1 Functionality: Text-Searching]

7.1.2 Multimedia Searching
[Fig: 2-Functionality: Multimedia Searching]

7.1.3 Searching Filter
[Fig: 3 – Functionality: Searching Filter]

7.1.4 Specific activity Search
[Fig: 4 – Functionality: Specific activity Search]
7.1.5 Functionality :
[Fig: 5 – Functionality]
7.2.1 Usability
[Fig: 6 Usability]
7.3.1 Performance:
[Fig: 7 Performance]
7.4.1 Reliability
[Fig: 8 Reliability]
7.5 Global preference
[Fig: 9 Global Preference]

8.1 Competitive System :
Attributes
Google Yahoo! Search Bing Ask alavista

1.1.1.1.1 Search with
one keyword 1 2 2 3 3
1.1.1.1.2 Search with
group of keywords 1 2 2 3 3
1.1.1.2.1 Search
exact statement 1 1 2 2 3
1.1.1.2.2 Search part
of statement 1 2 2 2 3
1.1.1.2.3 Case
sensitive Search 1 1 1 1 1
1.1.1.2.4 Ignore stop
words 1 1 1 1 1
(the,a,an,..etc)
1.1.1.3 Search with
different language 46 41 42 6 2
1.1.1.4.1 Search
numeric expression 2 2 2 2 2
in documents
1.1.1.4.2 Compute
numeric expression 30 4 20 6 4
1.1.1.5.1
Abbreviation Search 1 1 1 2 2
1.1.1.5.2 Synonyms
Search 1 1 1 2 2
1.1.1.5.3 Stemming
Search 1 1 2 2 2
1.1.1.5.4 Misspell
Corrected Search 1 1 1 2 2
1.1.1.6.1 Including
Search (using ‘+’ 1 2 2 1 2
operator)
1.1.1.6.2 Excluding
Search (using ‘-‘ 3 3 3 3 3
operator)
1.1.1.6.3
Combinational 2 3 3 3 3
Search (using ‘*’
operator)
1.1.1.6.4 Optional
Search (using ‘OR’ 1 3 1 2 2
operator)
1.1.2.1.1 Filename
search 2 2 2 3 3
1.1.2.1.2 Image link
search 2 2 2 2 2
1.1.2.1.3 Adjacent
text search 1 1 2 2 2
1.1.2.2.1 Popularity
based Search 1 2 1 2 2
1.1.2.2.2 Content
based Search 1 1 1 1 1
1.1.2.2.3 Content
Controlled Search 2 2 2 2 2
1.1.2.3 Audio
Searching 2 2 2 2 2
1.2.1.1 Pages with
Spam, doorway 1 1 1 2 1
1.2.1.2 Duplicate
Content 2 3 3 3 2
1.2.2.1 Adult content
filter 2 2 2 2 2
1.2.2.2 Casino
content filter 3 4 3 4 4
1.2.3.1 Pages related
to same content 3 3 2 3 3
1.2.3.2 Pages related
to same website 1 1 1 1 1
1.2.3.3 Linked page
2 2 2 2 2
1.2.4.1 RSS support
pages 2 3 2 3 3
1.2.4.2 Usage rights
1 2 1 2 2
1.2.4.3 Numeric
range filter 1 3 3 3 3
1.2.4.4 field to
search keyword 4 3 4 4 3
[title,text,URL,Link]
1.2.5 File specific
filter [pdf, word, 30 26 27 24 23
excel sheet,..]
1.2.6 Broken link
filter 4 3 3 4 3
1.2.7.1 Page created
time 4 3 3 4 3
1.2.7.2 Recent
update time 2 2 2 2 2
1.2.8.1 Location of
searching user [i.e. 2 3 2 4 4
weather forecast]
1.2.8.2 Location of
country 2 2 2 3 4
1.3.1 Weather
1 2 2 3 2
1.3.2 Blog
1 2 2 3 3
1.3.3 Movie time
3 1 2 3 3
1.3.4 Sport score
2 1 1 2 2
1.3.5 Stock price
2 1 2 2 3
1.3.6 Literature
[books,..] 9 9 9 8 8
1.3.7 Maps
10 8 9 7 7
2.1.1 Interface
visibility 10 9 9 9 9
2.1.2 Operability
10 10 10 9 9
2.1.3.1
Customization of 10 10 10 10 10
page size
2.1.3.2
Customization of 2 2 2 3 3
page rank
2.2.1 Result
visibility 10 8 9 8 8
2.2.2 Accessibility of
results 10 9 9 9 9
2.2.3 Availability of
cached result 2 2 2 3 2
2.3.1 Online help
1 1 1 2 1
2.3.2 Manual user-
guide 2 2 2 3 2
2.3.3 FAQ
30 28 30 27 28
2.3.4 Related
tutorial material 1 1 1 2 2
3.1.1 Load page time
2 2 2 2 2
3.1.2 Automatic
search suggestion 0.5 0.5 0.5 0.5 0.5
time
3.1.3.1 Time to
evaluate best result 3.5 3.5 3.5 4 4.5
3.1.3.2 Time to
evaluate top N result 3 3 3 3.5 4.0
3.2 Resource
Consumption 76 142 55 360 50
4.1.1 Popular pages
results 1 1 1 3 6
4.1.2 High rank
pages results 1 2 2 3 7
4.1.3 Coverage of
user need 2 3 3 4 3
4.2.1 Accuracy
90 75 85 70 75
4.2.2 Precision
95 80 85 80 80
4.2.3 Recall
88 78 82 78 78
4.2.4 Specificity
93 85 88 84 84
9.1 Results :
9.1.1 EVALUATION REPORT FOR THE SearchEngine PROJECT [SearchEngine.txt]
This report presents the evaluation results for the following 5 competitive
systems:
1. Google
2. Yahoo
3. bing
4. Ask
5. altavista
The evaluation is based on 71 elementary criteria grouped in the following 4

major groups:
1. Functionality
2. Usability
3. Performance
4. Reliability
This summary includes two parts: (1) System Comparison and Ranking, and
(2) Survey of Individual Systems. Deatailed numerical results can be found in
the report entitled "Detailed Evaluation Results of the SearchEngine Project".
(1) System Comparison and Ranking

---------------------------------
The global preference of a system indicates the global percentage of satisfied

requirements. Therefore, the best system has the highest global preference. The
ranking of competitive systems is based on decreasing global preferences, as
follows:
1. 94.89% Google
2. 89.78% bing
3. 88.18% Yahoo
4. 78.42% altavista
5. 77.35% Ask
Therefore, the best system is Google.

This system satisfies 94.89% of the requirements specified by evaluation criteria.
The absolute value of global preference depends both on the quality of each system
and the level of demand imposed by the evaluation criterion function. So, low global
preferences may sometimes reflect too demanding criteria. The relative ranking of
competitive systems is based on normalized preferences so that the best system has
the normalized global preference of 100%. Following is the ranking according to
normalized preferences:
1. 100.00% Google
2. 94.62% bing
3. 92.93% Yahoo
4. 82.65% altavista
5. 81.51% Ask
The relative differences between systems can be interpreted as follows:
System Google dominates system bing in 66.20% of inputs

System Google dominates system Yahoo in 72.54% of inputs
System Google dominates system altavista in 81.69% of inputs
System Google dominates system Ask in 83.80% of inputs
System bing domantes system Yahoo in 59.86% of inputs
The reasons for a specific value of global preference can be explained

by investigating the quality of all major components of the evaluated
systems. Following is the survey of preferences of 4 major system
components: Functionality, Usability, Performance, and Reliability.
Systems Functionality Usability Performance Reliability

-----------------------------------------------------------------------
Google 93.79 96.95 95.82 93.87
Yahoo 89.21 88.40 90.87 85.05
bing 90.12 88.86 97.22 86.18
Ask 82.26 85.55 63.49 73.33
altavis 81.20 88.10 96.62 60.97

-----------------------------------------------------------------------
(2) Survey of Individual Systems

--------------------------------
This survey highlights the strongest and the waekest components of

all evaluated systems. In particular, the survey includes lists of
the weakest components that are primary candidates for improvements.
This is an analysis of relative performance and for high quality
systems the weakest component can still satisfy a substantial
percentage of user's requiremenmts. Therefore, improvements are not
equally urgent for all systems. They are primariliy needed for systems
having a relatively low global preference.
Google
This system satisfies 94.89% of user's requirements. The best subsystem

of Google is Usability.
The best subsystem satisfies 96.95% of specified requirements.
The weakest subsystem of Google is Functionality.
The weakest subsystem satisfies 93.79% of specified requirements.
Weak components of this system are components that are rated

below the global preference. These are components that primarily
need improvement. Following is the sorted list of weak components,
starting with the weakest component:
ID X E[%] Elementary criterion

-------------------------------------------------------------
233 30.00 62.50 FAQ
1271 4.00 66.67 Page created time
126 4.00 66.67 Broken link filter
11162 3.00 77.78 Excluding Search (using ‘-‘ operator)
1222 3.00 77.78 Casino content filter
133 3.00 77.78 Movie time
1231 3.00 77.78 Pages related to same content
3132 3.00 86.09 Time to evaluate top N result
423 88.00 87.88 Recall
32 76.00 88.80 Resource Consumption
11163 2.00 88.89 Combinational Search (using ‘*’ operator)
1233 2.00 88.89 Linked page
1241 2.00 88.89 RSS support pages
11211 2.00 88.89 Filename search
11212 2.00 88.89 Image link search
1272 2.00 88.89 Recent update time
1281 2.00 88.89 Location of searching user [i.e. weather forecast]
1282 2.00 88.89 Location of country
11223 2.00 88.89 Content Controlled Search
134 2.00 88.89 Sport score
135 2.00 88.89 Stock price
136 9.00 88.89 Literature [books,..]
2132 2.00 88.89 Customization of page rank
223 2.00 88.89 Availability of cached result
232 2.00 88.89 Manual user-guide
1123 2.00 88.89 Audio Searching
1212 2.00 88.89 Duplicate Content
1221 2.00 88.89 Adult content filter
413 2.00 88.89 Coverage of user need
11141 2.00 88.89 Search numeric expression in documents
421 90.00 89.90 Accuracy
1113 46.00 90.38 Search with different languages
424 93.00 92.93 Specificity
Yahoo

of Yahoo is Performance.
The weakest subsystem of Yahoo is Reliability.


-------------------------------------------------------------
11142 4.00 0.00 Compute numeric expression
233 28.00 57.50 FAQ
421 75.00 74.75 Accuracy
1244 3.00 75.00 field to search keyword [title,text,URL,Link]
1243 3.00 77.78 Numeric range filter
137 8.00 77.78 Maps
221 8.00 77.78 Result visibility
11164 3.00 77.78 Optional Search (using ‘OR’ operator)
423 78.00 77.78 Recall
422 80.00 79.80 Precision
125 26.00 86.67 File specific filter [pdf, word, excel sheet,..]
bing

of bing is Performance.
The weakest subsystem of bing is Reliability.


-------------------------------------------------------------
233 30.00 62.50 FAQ
423 82.00 81.82 Recall
422 85.00 84.85 Precision
421 85.00 84.85 Accuracy
11111 2.00 88.89 Search with one keyword
1233 2.00 88.89 Linked page
11112 2.00 88.89 Search with group of keywords
11153 2.00 88.89 Stemming Search
11161 2.00 88.89 Including Search (using ‘+’ operator)
131 2.00 88.89 Weather
132 2.00 88.89 Blog
133 2.00 88.89 Movie time
135 2.00 88.89 Stock price
137 9.00 88.89 Maps
211 9.00 88.89 Interface visibility
222 9.00 88.89 Accessibility of results
11121 2.00 88.89 Search exact statement
11122 2.00 88.89 Search parts of the statement
412 2.00 88.89 High rank pages results
11213 2.00 88.89 Adjacent text search
Ask

of Ask is Usability.
The weakest subsystem of Ask is Performance.


-------------------------------------------------------------
233 27.00 55.00 FAQ
137 7.00 66.67 Maps
421 70.00 69.70 Accuracy
altavista

of altavista is Performance.
The weakest subsystem of altavista is Reliability.


-------------------------------------------------------------
411 6.00 44.44 Popular pages results
233 28.00 57.50 FAQ
137 7.00 66.67 Maps
421 75.00 74.75 Accuracy
132 3.00 77.78 Blog
133 3.00 77.78 Movie time
135 3.00 77.78 Stock price
423 78.00 77.78 Recall
9.2 DETAILED EVALUATION RESULTS FOR THE SearchEngine PROJECT
[SearchEngine.lst]
Competitive System(s):
1. Google
2. Yahoo
3. bing
4. Ask
5. altavista
PERFORMANCE VARIABLES:
11111. Search with one keyword

11112. Search with group of keywords
11121. Search exact statement
11122. Search parts of the statement
11123. Ignore stop words
11124. Case sensitive Search
1113. Search with different languages
11141. Search numeric expression in documents
11142. Compute numeric expression
11151. Abbreviation Search
11152. Synonyms Search
11153. Stemming Search
11154. Misspell Corrected Search
11161. Including Search (using ‘+’ operator)
11162. Excluding Search (using ‘-‘ operator)
11163. Combinational Search (using ‘*’ operator)
11164. Optional Search (using ‘OR’ operator)
11211. Filename search
11212. Image link search
11213. Adjacent text search
11221. Popularity based Search
11222. Content based Search
11223. Content Controlled Search
1123. Audio Searching
1211. Pages with Spam, doorway
1212. Duplicate Content
1221. Adult content filter
1222. Casino content filter
1231. Pages related to same content
1232. Pages related to same website
1233. Linked page
1241. RSS support pages
1242. Usage rights
1243. Numeric range filter
1244. field to search keyword [title,text,URL,Link]
125. File specific filter [pdf, word, excel sheet,..]
126. Broken link filter
1271. Page created time
1272. Recent update time
1281. Location of searching user [i.e. weather forecast]
1282. Location of country
131. Weather
132. Blog
133. Movie time
134. Sport score
135. Stock price
136. Literature [books,..]
137. Maps
211. Interface visibility
212. Operability
2131. Customization of page size
2132. Customization of page rank
221. Result visibility
222. Accessibility of results
223. Availability of cached result
231. Online help
232. Manual user-guide
233. FAQ
234. Related tutorial material
311. Load Page Time
312. Automatic search suggestion time
3131. Time to evaluate best result
3132. Time to evaluate top N result
32. Resource Consumption
411. Popular pages results
412. High rank pages results
413. Coverage of user need
421. Accuracy
422. Precision
423. Recall
424. Specificity
PERFORMANCE VARIABLES (INPUT VALUES)
# ID Google Yahoo bing Ask altavista

----------------------------------------------------------------
1 11111 1.000 2.000 2.000 3.000 3.000
2 11112 1.000 2.000 2.000 3.000 3.000
3 11121 1.000 1.000 2.000 2.000 3.000
4 11122 1.000 2.000 2.000 2.000 3.000
5 11123 1.000 1.000 1.000 1.000 1.000
6 11124 1.000 1.000 1.000 1.000 1.000
7 1113 46.000 41.000 42.000 6.000 2.000
8 11141 2.000 2.000 2.000 2.000 2.000
9 11142 30.000 4.000 20.000 6.000 4.000
10 11151 1.000 1.000 1.000 2.000 2.000
11 11152 1.000 1.000 1.000 2.000 2.000
12 11153 1.000 1.000 2.000 2.000 2.000
13 11154 1.000 1.000 1.000 2.000 2.000
14 11161 1.000 2.000 2.000 1.000 2.000
15 11162 3.000 3.000 3.000 3.000 3.000
16 11163 2.000 3.000 3.000 3.000 3.000
17 11164 1.000 3.000 1.000 2.000 2.000
18 11211 2.000 2.000 2.000 3.000 3.000
19 11212 2.000 2.000 2.000 2.000 2.000
20 11213 1.000 1.000 2.000 2.000 2.000
21 11221 1.000 2.000 1.000 2.000 2.000
22 11222 1.000 1.000 1.000 1.000 1.000
23 11223 2.000 2.000 2.000 2.000 2.000
24 1123 2.000 2.000 2.000 2.000 2.000
25 1211 1.000 1.000 1.000 2.000 1.000
26 1212 2.000 3.000 3.000 3.000 2.000
27 1221 2.000 2.000 2.000 2.000 2.000
28 1222 3.000 4.000 3.000 4.000 4.000
29 1231 3.000 3.000 2.000 3.000 3.000
30 1232 1.000 1.000 1.000 1.000 1.000
31 1233 2.000 2.000 2.000 2.000 2.000
32 1241 2.000 3.000 2.000 3.000 3.000
33 1242 1.000 2.000 1.000 2.000 2.000
34 1243 1.000 3.000 3.000 3.000 3.000
35 1244 4.000 3.000 4.000 4.000 3.000
36 125 30.000 26.000 27.000 24.000 23.000
37 126 4.000 3.000 3.000 4.000 3.000
38 1271 4.000 3.000 3.000 4.000 3.000
39 1272 2.000 2.000 2.000 2.000 2.000
40 1281 2.000 3.000 2.000 4.000 4.000
41 1282 2.000 2.000 2.000 3.000 4.000
42 131 1.000 2.000 2.000 3.000 2.000
43 132 1.000 2.000 2.000 3.000 3.000
44 133 3.000 1.000 2.000 3.000 3.000
45 134 2.000 1.000 1.000 2.000 2.000
46 135 2.000 1.000 2.000 2.000 3.000
47 136 9.000 9.000 9.000 8.000 8.000
48 137 10.000 8.000 9.000 7.000 7.000
49 211 10.000 9.000 9.000 9.000 9.000
50 212 10.000 10.000 10.000 9.000 9.000
51 2131 10.000 10.000 10.000 10.000 10.000
52 2132 2.000 2.000 2.000 3.000 3.000
53 221 10.000 8.000 9.000 8.000 8.000
54 222 10.000 9.000 9.000 9.000 9.000
55 223 2.000 2.000 2.000 3.000 2.000
56 231 1.000 1.000 1.000 2.000 1.000
57 232 2.000 2.000 2.000 3.000 2.000
58 233 30.000 28.000 30.000 27.000 28.000
59 234 1.000 1.000 1.000 2.000 2.000
60 311 2.000 2.000 2.000 2.000 2.000
61 312 0.500 0.500 0.500 0.500 0.500
62 3131 3.500 3.500 3.500 4.000 4.500
63 3132 3.000 3.000 3.000 3.500 4.000
64 32 76.000 142.000 55.000 360.000 50.000
65 411 1.000 1.000 1.000 3.000 6.000
66 412 1.000 2.000 2.000 3.000 7.000
67 413 2.000 3.000 3.000 4.000 3.000
68 421 90.000 75.000 85.000 70.000 75.000
69 422 95.000 80.000 85.000 80.000 80.000
70 423 88.000 78.000 82.000 78.000 78.000
71 424 93.000 85.000 88.000 84.000 84.000
ELEMENTARY PREFERENCES [%]

----------------------------------------------------------------
1 11111 100.00 88.89 88.89 77.78 77.78
2 11112 100.00 88.89 88.89 77.78 77.78
3 11121 100.00 100.00 88.89 88.89 77.78
4 11122 100.00 88.89 88.89 88.89 77.78
5 11123 100.00 100.00 100.00 100.00 100.00
6 11124 100.00 100.00 100.00 100.00 100.00
7 1113 90.38 80.77 82.69 10.42 2.08
8 11141 88.89 88.89 88.89 88.89 88.89
9 11142 100.00 0.00 66.67 9.09 0.00
10 11151 100.00 100.00 100.00 88.89 88.89
11 11152 100.00 100.00 100.00 88.89 88.89
12 11153 100.00 100.00 88.89 88.89 88.89
13 11154 100.00 100.00 100.00 88.89 88.89
14 11161 100.00 88.89 88.89 100.00 88.89
15 11162 77.78 77.78 77.78 77.78 77.78
16 11163 88.89 77.78 77.78 77.78 77.78
17 11164 100.00 77.78 100.00 88.89 88.89
18 11211 88.89 88.89 88.89 77.78 77.78
19 11212 88.89 88.89 88.89 88.89 88.89
20 11213 100.00 100.00 88.89 88.89 88.89
21 11221 100.00 88.89 100.00 88.89 88.89
22 11222 100.00 100.00 100.00 100.00 100.00
23 11223 88.89 88.89 88.89 88.89 88.89
24 1123 88.89 88.89 88.89 88.89 88.89
25 1211 100.00 100.00 100.00 88.89 100.00
26 1212 88.89 77.78 77.78 77.78 88.89
27 1221 88.89 88.89 88.89 88.89 88.89
28 1222 77.78 66.67 77.78 66.67 66.67
29 1231 77.78 77.78 88.89 77.78 77.78
30 1232 100.00 100.00 100.00 100.00 100.00
31 1233 88.89 88.89 88.89 88.89 88.89
32 1241 88.89 77.78 88.89 77.78 77.78
33 1242 100.00 88.89 100.00 88.89 88.89
34 1243 100.00 77.78 77.78 77.78 77.78
35 1244 100.00 75.00 100.00 100.00 75.00
36 125 100.00 86.67 90.00 80.00 76.67
37 126 66.67 77.78 77.78 66.67 77.78
38 1271 66.67 77.78 77.78 66.67 77.78
39 1272 88.89 88.89 88.89 88.89 88.89
40 1281 88.89 77.78 88.89 66.67 66.67
41 1282 88.89 88.89 88.89 77.78 66.67
42 131 100.00 88.89 88.89 77.78 88.89
43 132 100.00 88.89 88.89 77.78 77.78
44 133 77.78 100.00 88.89 77.78 77.78
45 134 88.89 100.00 100.00 88.89 88.89
46 135 88.89 100.00 88.89 88.89 77.78
47 136 88.89 88.89 88.89 77.78 77.78
48 137 100.00 77.78 88.89 66.67 66.67
49 211 100.00 88.89 88.89 88.89 88.89
50 212 100.00 100.00 100.00 88.89 88.89
51 2131 100.00 100.00 100.00 100.00 100.00
52 2132 88.89 88.89 88.89 77.78 77.78
53 221 100.00 77.78 88.89 77.78 77.78
54 222 100.00 88.89 88.89 88.89 88.89
55 223 88.89 88.89 88.89 77.78 88.89
56 231 100.00 100.00 100.00 88.89 100.00
57 232 88.89 88.89 88.89 77.78 88.89
58 233 62.50 57.50 62.50 55.00 57.50
59 234 100.00 100.00 100.00 88.89 88.89
60 311 100.00 100.00 100.00 100.00 100.00
61 312 100.00 100.00 100.00 100.00 100.00
62 3131 100.00 100.00 100.00 100.00 96.43
63 3132 86.09 86.09 86.09 82.78 79.47
64 32 88.80 75.60 93.00 32.00 94.00
65 411 100.00 100.00 100.00 77.78 44.44
66 412 100.00 88.89 88.89 77.78 33.33
67 413 88.89 77.78 77.78 66.67 77.78
68 421 89.90 74.75 84.85 69.70 74.75
69 422 94.95 79.80 84.85 79.80 79.80
70 423 87.88 77.78 81.82 77.78 77.78
71 424 92.93 84.85 87.88 83.84 83.84
ALL PREFERENCES [%]

----------------------------------------------------------------
1 11111 100.00 88.89 88.89 77.78 77.78
2 11112 100.00 88.89 88.89 77.78 77.78
3 1111 100.00 88.89 88.89 77.78 77.78
4 11121 100.00 100.00 88.89 88.89 77.78
5 11122 100.00 88.89 88.89 88.89 77.78
6 1000 100.00 92.65 88.89 88.89 77.78
7 11123 100.00 100.00 100.00 100.00 100.00
8 11124 100.00 100.00 100.00 100.00 100.00
9 1001 100.00 100.00 100.00 100.00 100.00
10 1002 100.00 93.62 90.35 90.35 80.69
11 1003 100.00 90.50 89.40 81.82 78.78
12 1113 90.38 80.77 82.69 10.42 2.08
13 11141 88.89 88.89 88.89 88.89 88.89
14 11142 100.00 0.00 66.67 9.09 0.00
15 1114 91.07 61.99 84.25 68.20 61.99
16 11151 100.00 100.00 100.00 88.89 88.89
17 11152 100.00 100.00 100.00 88.89 88.89
18 11153 100.00 100.00 88.89 88.89 88.89
19 11154 100.00 100.00 100.00 88.89 88.89
20 1115 100.00 100.00 97.73 88.89 88.89
21 11161 100.00 88.89 88.89 100.00 88.89
22 11162 77.78 77.78 77.78 77.78 77.78
23 11163 88.89 77.78 77.78 77.78 77.78
24 11164 100.00 77.78 100.00 88.89 88.89
25 1116 95.98 83.81 88.19 92.00 86.05
26 1004 95.49 85.31 90.12 69.68 65.58
27 111 99.29 89.68 89.51 79.87 76.65
28 11211 88.89 88.89 88.89 77.78 77.78
29 11212 88.89 88.89 88.89 88.89 88.89
30 11213 100.00 100.00 88.89 88.89 88.89
31 1121 94.16 94.16 88.89 86.45 86.45
32 11221 100.00 88.89 100.00 88.89 88.89
33 11222 100.00 100.00 100.00 100.00 100.00
34 1005 100.00 94.16 100.00 94.16 94.16
35 11223 88.89 88.89 88.89 88.89 88.89
36 1122 97.96 93.20 97.96 93.20 93.20
37 1123 88.89 88.89 88.89 88.89 88.89
38 112 93.97 92.15 92.34 89.80 89.80
39 1211 100.00 100.00 100.00 88.89 100.00
40 1212 88.89 77.78 77.78 77.78 88.89
41 121 94.94 89.74 89.74 83.82 94.94
42 1221 88.89 88.89 88.89 88.89 88.89
43 1222 77.78 66.67 77.78 66.67 66.67
44 122 87.66 86.12 87.66 86.12 86.12
45 1231 77.78 77.78 88.89 77.78 77.78
46 1232 100.00 100.00 100.00 100.00 100.00
47 1233 88.89 88.89 88.89 88.89 88.89
48 123 87.54 87.54 92.71 87.54 87.54
49 1241 88.89 77.78 88.89 77.78 77.78
50 1242 100.00 88.89 100.00 88.89 88.89
51 1243 100.00 77.78 77.78 77.78 77.78
52 1244 100.00 75.00 100.00 100.00 75.00
53 1006 96.05 81.04 88.14 81.04 81.04
54 124 98.19 77.62 94.34 90.62 77.62
55 1007 90.86 85.95 90.38 86.53 87.11
56 125 100.00 86.67 90.00 80.00 76.67
57 126 66.67 77.78 77.78 66.67 77.78
58 1271 66.67 77.78 77.78 66.67 77.78
59 1272 88.89 88.89 88.89 88.89 88.89
60 127 84.25 86.62 86.62 84.25 86.62
61 1281 88.89 77.78 88.89 66.67 66.67
62 1282 88.89 88.89 88.89 77.78 66.67
63 128 88.89 85.50 88.89 74.38 66.67
64 1008 86.73 84.98 86.75 77.74 77.27
65 12 89.35 85.61 89.07 83.16 83.30
66 131 100.00 88.89 88.89 77.78 88.89
67 132 100.00 88.89 88.89 77.78 77.78
68 133 77.78 100.00 88.89 77.78 77.78
69 134 88.89 100.00 100.00 88.89 88.89
70 135 88.89 100.00 88.89 88.89 77.78
71 136 88.89 88.89 88.89 77.78 77.78
72 137 100.00 77.78 88.89 66.67 66.67
73 1009 90.83 93.27 88.89 77.78 81.59
74 1010 92.71 89.21 91.06 78.06 75.92
75 13 92.05 90.62 90.30 77.96 77.89
76 11 97.10 90.65 90.62 83.61 81.49
77 1011 94.26 88.83 90.07 83.45 82.12
78 1 93.79 89.21 90.12 82.26 81.20
79 211 100.00 88.89 88.89 88.89 88.89
80 212 100.00 100.00 100.00 88.89 88.89
81 1012 100.00 95.13 95.13 88.89 88.89
82 2131 100.00 100.00 100.00 100.00 100.00
83 2132 88.89 88.89 88.89 77.78 77.78
84 213 91.07 91.07 91.07 82.05 82.05
85 21 98.81 94.59 94.59 87.98 87.98
86 221 100.00 77.78 88.89 77.78 77.78
87 222 100.00 88.89 88.89 88.89 88.89
88 223 88.89 88.89 88.89 77.78 88.89
89 1013 97.74 88.89 88.89 86.62 88.89
90 231 100.00 100.00 100.00 88.89 100.00
91 232 88.89 88.89 88.89 77.78 88.89
92 233 62.50 57.50 62.50 55.00 57.50
93 234 100.00 100.00 100.00 88.89 88.89
94 23 88.60 87.41 88.60 78.25 84.22
95 1014 92.71 85.78 88.72 79.83 83.86
96 2 96.95 88.40 88.86 85.55 88.10
97 311 100.00 100.00 100.00 100.00 100.00
98 312 100.00 100.00 100.00 100.00 100.00
99 3131 100.00 100.00 100.00 100.00 96.43
100 3132 86.09 86.09 86.09 82.78 79.47
101 1015 100.00 100.00 100.00 100.00 100.00
102 1016 97.16 97.16 97.16 96.46 92.94
103 31 99.13 99.13 99.13 98.91 97.79
104 32 88.80 75.60 93.00 32.00 94.00
105 3 95.82 90.87 97.22 63.49 96.62
106 411 100.00 100.00 100.00 77.78 44.44
107 412 100.00 88.89 88.89 77.78 33.33
108 413 88.89 77.78 77.78 66.67 77.78
109 41 94.37 86.44 86.44 72.13 57.47
110 421 89.90 74.75 84.85 69.70 74.75
111 422 94.95 79.80 84.85 79.80 79.80
112 423 87.88 77.78 81.82 77.78 77.78
113 424 92.93 84.85 87.88 83.84 83.84
114 42 91.90 79.86 85.14 78.51 79.57
115 4 93.87 85.05 86.18 73.33 60.97
116 9999 94.89 88.18 89.78 77.35 78.42
Elementary Preferences for System = Google

-------------------------------------------------------------
11123 1.00 100.00 Ignore stop words
11124 1.00 100.00 Case sensitive Search
11151 1.00 100.00 Abbreviation Search
11152 1.00 100.00 Synonyms Search
11154 1.00 100.00 Misspell Corrected Search
11221 1.00 100.00 Popularity based Search
11222 1.00 100.00 Content based Search
1211 1.00 100.00 Pages with Spam, doorway
1232 1.00 100.00 Pages related to same website
1233 2.00 88.89 Linked page
1242 1.00 100.00 Usage rights
131 1.00 100.00 Weather
132 1.00 100.00 Blog
133 3.00 77.78 Movie time
134 2.00 88.89 Sport score
135 2.00 88.89 Stock price
137 10.00 100.00 Maps
212 10.00 100.00 Operability
2131 10.00 100.00 Customization of page size
231 1.00 100.00 Online help
233 30.00 62.50 FAQ
234 1.00 100.00 Related tutorial material
311 2.00 100.00 Load Page Time
312 0.50 100.00 Automatic search suggestion time
3131 3.50 100.00 Time to evaluate best result
421 90.00 89.90 Accuracy
422 95.00 94.95 Precision
423 88.00 87.88 Recall
Sorted Elementary Preferences for System = Google

-------------------------------------------------------------
233 30.00 62.50 FAQ
133 3.00 77.78 Movie time
423 88.00 87.88 Recall
1233 2.00 88.89 Linked page
134 2.00 88.89 Sport score
135 2.00 88.89 Stock price
421 90.00 89.90 Accuracy
422 95.00 94.95 Precision

131 1.00 100.00 Weather
132 1.00 100.00 Blog
137 10.00 100.00 Maps
212 10.00 100.00 Operability
231 1.00 100.00 Online help
1242 1.00 100.00 Usage rights
Elementary Preferences for System = Yahoo

-------------------------------------------------------------
1233 2.00 88.89 Linked page
1242 2.00 88.89 Usage rights
131 2.00 88.89 Weather
132 2.00 88.89 Blog
133 1.00 100.00 Movie time
134 1.00 100.00 Sport score
135 1.00 100.00 Stock price
137 8.00 77.78 Maps
212 10.00 100.00 Operability
231 1.00 100.00 Online help
233 28.00 57.50 FAQ
421 75.00 74.75 Accuracy
422 80.00 79.80 Precision
423 78.00 77.78 Recall
Sorted Elementary Preferences for System = Yahoo

-------------------------------------------------------------
233 28.00 57.50 FAQ
421 75.00 74.75 Accuracy
137 8.00 77.78 Maps
423 78.00 77.78 Recall
422 80.00 79.80 Precision

131 2.00 88.89 Weather
132 2.00 88.89 Blog
1233 2.00 88.89 Linked page
1242 2.00 88.89 Usage rights
212 10.00 100.00 Operability
231 1.00 100.00 Online help
133 1.00 100.00 Movie time
134 1.00 100.00 Sport score
135 1.00 100.00 Stock price
Elementary Preferences for System = bing

-------------------------------------------------------------
1233 2.00 88.89 Linked page
1242 1.00 100.00 Usage rights
131 2.00 88.89 Weather
132 2.00 88.89 Blog
133 2.00 88.89 Movie time
134 1.00 100.00 Sport score
135 2.00 88.89 Stock price
137 9.00 88.89 Maps
212 10.00 100.00 Operability
231 1.00 100.00 Online help
233 30.00 62.50 FAQ
421 85.00 84.85 Accuracy
422 85.00 84.85 Precision
423 82.00 81.82 Recall
Sorted Elementary Preferences for System = bing

-------------------------------------------------------------
233 30.00 62.50 FAQ
423 82.00 81.82 Recall
422 85.00 84.85 Precision
421 85.00 84.85 Accuracy
1233 2.00 88.89 Linked page
131 2.00 88.89 Weather
132 2.00 88.89 Blog
133 2.00 88.89 Movie time
135 2.00 88.89 Stock price
137 9.00 88.89 Maps
231 1.00 100.00 Online help
134 1.00 100.00 Sport score
1242 1.00 100.00 Usage rights
212 10.00 100.00 Operability
Elementary Preferences for System = Ask

-------------------------------------------------------------
1233 2.00 88.89 Linked page
1242 2.00 88.89 Usage rights
131 3.00 77.78 Weather
132 3.00 77.78 Blog
133 3.00 77.78 Movie time
134 2.00 88.89 Sport score
135 2.00 88.89 Stock price
137 7.00 66.67 Maps
212 9.00 88.89 Operability
231 2.00 88.89 Online help
233 27.00 55.00 FAQ
421 70.00 69.70 Accuracy
422 80.00 79.80 Precision
423 78.00 77.78 Recall
Sorted Elementary Preferences for System = Ask

-------------------------------------------------------------
233 27.00 55.00 FAQ
137 7.00 66.67 Maps
421 70.00 69.70 Accuracy

131 3.00 77.78 Weather
132 3.00 77.78 Blog
133 3.00 77.78 Movie time
423 78.00 77.78 Recall
422 80.00 79.80 Precision
134 2.00 88.89 Sport score
135 2.00 88.89 Stock price
231 2.00 88.89 Online help
1233 2.00 88.89 Linked page
1242 2.00 88.89 Usage rights
Elementary Preferences for System = altavista

-------------------------------------------------------------
1233 2.00 88.89 Linked page
1242 2.00 88.89 Usage rights
131 2.00 88.89 Weather
132 3.00 77.78 Blog
133 3.00 77.78 Movie time
134 2.00 88.89 Sport score
135 3.00 77.78 Stock price
137 7.00 66.67 Maps
231 1.00 100.00 Online help
233 28.00 57.50 FAQ
421 75.00 74.75 Accuracy
422 80.00 79.80 Precision
423 78.00 77.78 Recall
Sorted Elementary Preferences for System = altavista

-------------------------------------------------------------
233 28.00 57.50 FAQ
137 7.00 66.67 Maps
421 75.00 74.75 Accuracy
132 3.00 77.78 Blog
133 3.00 77.78 Movie time
135 3.00 77.78 Stock price
423 78.00 77.78 Recall

422 80.00 79.80 Precision
131 2.00 88.89 Weather
134 2.00 88.89 Sport score
1233 2.00 88.89 Linked page
1242 2.00 88.89 Usage rights
231 1.00 100.00 Online help
Subsystem Preferences for System = Google
ID E[%] Subsystem
-------------------------------------------------------------
1111 100.00 Keyword Searching
1000 100.00 Temp GCD for 11121 11122
1001 100.00 Temp GCD for 11123 11124
1002 100.00 Temp GCD for 1000 1001
1003 100.00 Temp GCD for 1111 1002
1114 91.07 Numeric Expression Search
1115 100.00 Relational Searching
1116 95.98 Operator Searching
1004 95.49 Temp GCD for 1113 1114 1115 1116
111 99.29 Text Searching
1121 94.16 Image Searching
1005 100.00 * Temp GCD for 11221 11222
1122 97.96 Video Searching
112 93.97 Multimedia Search
121 94.94 Security filter
122 87.66 Citation filter
123 87.54 Domain filter
1006 96.05 * 1006
124 98.19 Extended filter
1007 90.86 *Temp GCD for 122 123 124
127 84.25 Time filter
128 88.89 Location filter
1008 86.73 *Temp GCD for 125 126 127 128
12 89.35 Searching Filter
1009 90.83 Temp GCD for 131 132 133
1010 92.71 Temp GCD for 134 135 136 137
13 92.05 Specific activity Search
11 97.10 Searching Input
1011 94.26 Temp GCD for 11 12
1 93.79 Functionality
1012 100.00 Temp GCD for 211 212
213 91.07 Customization
21 98.81 Interface usability
1013 97.74 Temp GCD for 222 223
23 88.60 User guide
1014 92.71 Temp GCD for 221 1013 23
2 96.95 Usability
1015 100.00 Temp GCD for 311 312
1016 97.16 3131 3132
31 99.13 21
3 95.82 Performance
41 94.37 User satisfaction
42 91.90 Confusion matrix
4 93.87 Reliability
9999 94.89 Whole_Search_Engine
Sorted Subsystem Preferences for System = Google

ID E[%] Subsystem
-------------------------------------------------------------
1008 86.73 *Temp GCD for 125 126 127 128
23 88.60 User guide
1009 90.83 Temp GCD for 131 132 133
1007 90.86 *Temp GCD for 122 123 124
1014 92.71 Temp GCD for 221 1013 23
1010 92.71 Temp GCD for 134 135 136 137
4 93.87 Reliability
1011 94.26 Temp GCD for 11 12

1004 95.49 Temp GCD for 1113 1114 1115 1116
3 95.82 Performance
1006 96.05 * 1006
2 96.95 Usability
1016 97.16 3131 3132
1013 97.74 Temp GCD for 222 223
31 99.13 21
1000 100.00 Temp GCD for 11121 11122
1015 100.00 Temp GCD for 311 312
1001 100.00 Temp GCD for 11123 11124
1012 100.00 Temp GCD for 211 212
1002 100.00 Temp GCD for 1000 1001
1003 100.00 Temp GCD for 1111 1002
1005 100.00 * Temp GCD for 11221 11222
Subsystem Preferences for System = Yahoo
ID E[%] Subsystem
-------------------------------------------------------------
1000 92.65 Temp GCD for 11121 11122
1001 100.00 Temp GCD for 11123 11124
1002 93.62 Temp GCD for 1000 1001
1003 90.50 Temp GCD for 1111 1002
1004 85.31 Temp GCD for 1113 1114 1115 1116
1005 94.16 * Temp GCD for 11221 11222
1006 81.04 * 1006
1007 85.95 *Temp GCD for 122 123 124
1008 84.98 *Temp GCD for 125 126 127 128
1009 93.27 Temp GCD for 131 132 133
1010 89.21 Temp GCD for 134 135 136 137
1011 88.83 Temp GCD for 11 12
1012 95.13 Temp GCD for 211 212
1013 88.89 Temp GCD for 222 223
23 87.41 User guide
1014 85.78 Temp GCD for 221 1013 23
2 88.40 Usability
1015 100.00 Temp GCD for 311 312
1016 97.16 3131 3132
31 99.13 21
3 90.87 Performance
4 85.05 Reliability
Sorted Subsystem Preferences for System = Yahoo
ID E[%] Subsystem
-------------------------------------------------------------
1006 81.04 * 1006
1008 84.98 *Temp GCD for 125 126 127 128
4 85.05 Reliability
1004 85.31 Temp GCD for 1113 1114 1115 1116
1014 85.78 Temp GCD for 221 1013 23
1007 85.95 *Temp GCD for 122 123 124
23 87.41 User guide

2 88.40 Usability
1011 88.83 Temp GCD for 11 12
1013 88.89 Temp GCD for 222 223
1010 89.21 Temp GCD for 134 135 136 137
1003 90.50 Temp GCD for 1111 1002
3 90.87 Performance
1000 92.65 Temp GCD for 11121 11122
1009 93.27 Temp GCD for 131 132 133
1002 93.62 Temp GCD for 1000 1001
1005 94.16 * Temp GCD for 11221 11222
1012 95.13 Temp GCD for 211 212
1016 97.16 3131 3132
31 99.13 21
1015 100.00 Temp GCD for 311 312
1001 100.00 Temp GCD for 11123 11124
Subsystem Preferences for System = bing
ID E[%] Subsystem
-------------------------------------------------------------
1000 88.89 Temp GCD for 11121 11122
1001 100.00 Temp GCD for 11123 11124
1002 90.35 Temp GCD for 1000 1001
1003 89.40 Temp GCD for 1111 1002
1004 90.12 Temp GCD for 1113 1114 1115 1116
1005 100.00 * Temp GCD for 11221 11222
1006 88.14 * 1006
1007 90.38 *Temp GCD for 122 123 124
1008 86.75 *Temp GCD for 125 126 127 128
1009 88.89 Temp GCD for 131 132 133
1010 91.06 Temp GCD for 134 135 136 137
1011 90.07 Temp GCD for 11 12
1012 95.13 Temp GCD for 211 212
1013 88.89 Temp GCD for 222 223
23 88.60 User guide
1014 88.72 Temp GCD for 221 1013 23
2 88.86 Usability
1015 100.00 Temp GCD for 311 312
1016 97.16 3131 3132
31 99.13 21
3 97.22 Performance
4 86.18 Reliability
Sorted Subsystem Preferences for System = bing
ID E[%] Subsystem
-------------------------------------------------------------
4 86.18 Reliability
1008 86.75 *Temp GCD for 125 126 127 128
1006 88.14 * 1006
23 88.60 User guide
1014 88.72 Temp GCD for 221 1013 23
2 88.86 Usability
1000 88.89 Temp GCD for 11121 11122
1009 88.89 Temp GCD for 131 132 133
1013 88.89 Temp GCD for 222 223
1003 89.40 Temp GCD for 1111 1002
1011 90.07 Temp GCD for 11 12

1004 90.12 Temp GCD for 1113 1114 1115 1116
1002 90.35 Temp GCD for 1000 1001
1007 90.38 *Temp GCD for 122 123 124
1010 91.06 Temp GCD for 134 135 136 137
1012 95.13 Temp GCD for 211 212
1016 97.16 3131 3132
3 97.22 Performance
31 99.13 21
1001 100.00 Temp GCD for 11123 11124
1015 100.00 Temp GCD for 311 312
1005 100.00 * Temp GCD for 11221 11222
Subsystem Preferences for System = Ask
ID E[%] Subsystem
-------------------------------------------------------------
1000 88.89 Temp GCD for 11121 11122
1001 100.00 Temp GCD for 11123 11124
1002 90.35 Temp GCD for 1000 1001
1003 81.82 Temp GCD for 1111 1002
1004 69.68 Temp GCD for 1113 1114 1115 1116
1005 94.16 * Temp GCD for 11221 11222
1006 81.04 * 1006
1007 86.53 *Temp GCD for 122 123 124
1008 77.74 *Temp GCD for 125 126 127 128
1009 77.78 Temp GCD for 131 132 133
1010 78.06 Temp GCD for 134 135 136 137
1011 83.45 Temp GCD for 11 12
1012 88.89 Temp GCD for 211 212
1013 86.62 Temp GCD for 222 223
23 78.25 User guide
1014 79.83 Temp GCD for 221 1013 23
2 85.55 Usability
1015 100.00 Temp GCD for 311 312
1016 96.46 3131 3132
31 98.91 21
3 63.49 Performance
4 73.33 Reliability
Sorted Subsystem Preferences for System = Ask
ID E[%] Subsystem
-------------------------------------------------------------
3 63.49 Performance
1004 69.68 Temp GCD for 1113 1114 1115 1116
4 73.33 Reliability
1008 77.74 *Temp GCD for 125 126 127 128

1009 77.78 Temp GCD for 131 132 133
1010 78.06 Temp GCD for 134 135 136 137
23 78.25 User guide
1014 79.83 Temp GCD for 221 1013 23
1006 81.04 * 1006
1003 81.82 Temp GCD for 1111 1002
1011 83.45 Temp GCD for 11 12
2 85.55 Usability
1007 86.53 *Temp GCD for 122 123 124
1013 86.62 Temp GCD for 222 223
1012 88.89 Temp GCD for 211 212
1000 88.89 Temp GCD for 11121 11122
1002 90.35 Temp GCD for 1000 1001
1005 94.16 * Temp GCD for 11221 11222
1016 96.46 3131 3132
31 98.91 21
1015 100.00 Temp GCD for 311 312
1001 100.00 Temp GCD for 11123 11124
Subsystem Preferences for System = altavista
ID E[%] Subsystem
-------------------------------------------------------------
1000 77.78 Temp GCD for 11121 11122
1001 100.00 Temp GCD for 11123 11124
1002 80.69 Temp GCD for 1000 1001
1003 78.78 Temp GCD for 1111 1002
1004 65.58 Temp GCD for 1113 1114 1115 1116
1005 94.16 * Temp GCD for 11221 11222
1006 81.04 * 1006
1007 87.11 *Temp GCD for 122 123 124
1008 77.27 *Temp GCD for 125 126 127 128
1009 81.59 Temp GCD for 131 132 133
1010 75.92 Temp GCD for 134 135 136 137
1011 82.12 Temp GCD for 11 12
1012 88.89 Temp GCD for 211 212
1013 88.89 Temp GCD for 222 223
23 84.22 User guide
1014 83.86 Temp GCD for 221 1013 23
2 88.10 Usability
1015 100.00 Temp GCD for 311 312
1016 92.94 3131 3132
31 97.79 21
3 96.62 Performance
4 60.97 Reliability
Sorted Subsystem Preferences for System = altavista
ID E[%] Subsystem
-------------------------------------------------------------
4 60.97 Reliability
1004 65.58 Temp GCD for 1113 1114 1115 1116
1010 75.92 Temp GCD for 134 135 136 137
1008 77.27 *Temp GCD for 125 126 127 128
1000 77.78 Temp GCD for 11121 11122
1003 78.78 Temp GCD for 1111 1002

1002 80.69 Temp GCD for 1000 1001
1006 81.04 * 1006
1009 81.59 Temp GCD for 131 132 133
1011 82.12 Temp GCD for 11 12
1014 83.86 Temp GCD for 221 1013 23
23 84.22 User guide
1007 87.11 *Temp GCD for 122 123 124
2 88.10 Usability
1012 88.89 Temp GCD for 211 212
1013 88.89 Temp GCD for 222 223
1016 92.94 3131 3132
1005 94.16 * Temp GCD for 11221 11222
3 96.62 Performance
31 97.79 21
1015 100.00 Temp GCD for 311 312
1001 100.00 Temp GCD for 11123 11124
Note: All sorted reports are divided in two parts: the first part contains
components that are below the average quality level for the analyzed system
and the second part contains those components that are above the average
quality level
9.3 SearchEngine.res
Google
100 100 100 100 100 100 100 100 100 100
100 90.3846 88.8889 100 91.0711 100 100 100 100 100
100 77.7778 88.8889 100 95.978 95.4867 99.2907 88.8889 88.8889 100
94.1613 100 100 100 88.8889 97.959 88.8889 93.967 100 88.8889
94.9383 88.8889 77.7778 87.6579 77.7778 100 88.8889 87.5421 88.8889 100
100 100 96.0476 98.1875 90.8575 100 66.6667 66.6667 88.8889 84.2513
88.8889 88.8889 88.8889 86.7288 89.3539 100 100 77.7778 88.8889 88.8889
88.8889 100 90.8257 92.7102 92.0489 97.1004 94.2619 93.7855 100 100
100 100 88.8889 91.0711 98.8115 100 100 88.8889 97.738 100
88.8889 62.5 100 88.5995 92.7073 96.9464 100 100 100 86.0927
100 97.1553 99.1317 88.8 95.8232 100 100 88.8889 94.3746 89.899
94.9495 87.8788 92.9293 91.9015 93.8709 94.8858
Yahoo
88.8889 88.8889 88.8889 100 88.8889 92.6521 100 100 100 93.6205
90.4972 80.7692 88.8889 0 61.9853 100 100 100 100 100
88.8889 77.7778 77.7778 77.7778 83.8056 85.312 89.6803 88.8889 88.8889 100
94.1613 88.8889 100 94.1633 88.8889 93.1976 88.8889 92.1478 100 77.7778
89.7373 88.8889 66.6667 86.1237 77.7778 100 88.8889 87.5421 77.7778 88.8889
77.7778 75 81.0445 77.6207 85.9511 86.6667 77.7778 77.7778 88.8889 86.6216
77.7778 88.8889 85.4964 84.9819 85.6086 88.8889 88.8889 100 100 100
88.8889 77.7778 93.2663 89.2081 90.6206 90.6536 88.8309 89.214 88.8889 100
95.1324 100 88.8889 91.0711 94.5942 77.7778 88.8889 88.8889 88.8889 100
88.8889 57.5 100 87.4093 85.7789 88.4011 100 100 100 86.0927
100 97.1553 99.1317 75.6 90.8727 100 88.8889 77.7778 86.4394 74.7475
79.798 77.7778 84.8485 79.8605 85.0506 88.18
bing
88.8889 88.8889 88.8889 88.8889 88.8889 88.8889 100 100 100 90.3518
89.3963 82.6923 88.8889 66.6667 84.2513 100 100 88.8889 100 97.7303
88.8889 77.7778 77.7778 100 88.1911 90.1191 89.5087 88.8889 88.8889 88.8889
88.8889 100 100 100 88.8889 97.959 88.8889 92.3353 100 77.7778
89.7373 88.8889 77.7778 87.6579 88.8889 100 88.8889 92.7142 88.8889 100
77.7778 100 88.1402 94.3431 90.384 90 77.7778 77.7778 88.8889 86.6216
88.8889 88.8889 88.8889 86.7543 89.0683 88.8889 88.8889 88.8889 100 88.8889
88.8889 88.8889 88.8889 91.0635 90.3001 90.6213 90.0725 90.1213 88.8889 100
95.1324 100 88.8889 91.0711 94.5942 88.8889 88.8889 88.8889 88.8889 100
88.8889 62.5 100 88.5995 88.7152 88.8618 100 100 100 86.0927
100 97.1553 99.1317 93 97.2208 100 88.8889 77.7778 86.4394 84.8485
84.8485 81.8182 87.8788 85.1395 86.1767 89.7764
Ask
77.7778 77.7778 77.7778 88.8889 88.8889 88.8889 100 100 100 90.3518
81.8171 10.4167 88.8889 9.09091 68.2024 88.8889 88.8889 88.8889 88.8889 88.8889
100 77.7778 77.7778 88.8889 91.9989 69.6791 79.8701 77.7778 88.8889 88.8889
86.4543 88.8889 100 94.1633 88.8889 93.1976 88.8889 89.803 88.8889 77.7778
83.819 88.8889 66.6667 86.1237 77.7778 100 88.8889 87.5421 77.7778 88.8889
77.7778 100 81.0445 90.6169 86.5303 80 66.6667 66.6667 88.8889 84.2513
66.6667 77.7778 74.3761 77.7405 83.1634 77.7778 77.7778 77.7778 88.8889 88.8889
77.7778 66.6667 77.7778 78.0629 77.9631 83.6054 83.4502 82.2622 88.8889 88.8889
88.8889 100 77.7778 82.0512 87.9798 77.7778 88.8889 77.7778 86.6216 88.8889
77.7778 55 88.8889 78.2542 79.8324 85.5476 100 100 100 82.7815
100 96.4576 98.9141 32 63.4887 77.7778 77.7778 66.6667 72.1309 69.697
79.798 77.7778 83.8384 78.5127 73.3343 77.3459
altavista
77.7778 77.7778 77.7778 77.7778 77.7778 77.7778 100 100 100 80.6944
78.7777 2.08333 88.8889 0 61.9853 88.8889 88.8889 88.8889 88.8889 88.8889
88.8889 77.7778 77.7778 88.8889 86.0481 65.5789 76.6524 77.7778 88.8889 88.8889
86.4543 88.8889 100 94.1633 88.8889 93.1976 88.8889 89.803 100 88.8889
94.9383 88.8889 66.6667 86.1237 77.7778 100 88.8889 87.5421 77.7778 88.8889
77.7778 75 81.0445 77.6207 87.1146 76.6667 77.7778 77.7778 88.8889 86.6216
66.6667 66.6667 66.6667 77.2685 83.3033 88.8889 77.7778 77.7778 88.8889 77.7778
77.7778 66.6667 81.5946 75.9182 77.8871 81.4889 82.1161 81.2023 88.8889 88.8889
88.8889 100 77.7778 82.0512 87.9798 77.7778 88.8889 88.8889 88.8889 100
88.8889 57.5 88.8889 84.2168 83.8634 88.0973 100 100 96.4286 79.4702
100 92.9375 97.7867 94 96.6235 44.4444 33.3333 77.7778 57.4659 74.7475
79.798 77.7778 83.8384 79.5652 60.9744 78.4239
9.4 SUMMARY OF RESULTS FOR THE SearchEngine PROJECT
[SearchEngine.sum]
This report presents the final evaluation results (global preferences)

for the SearchEngine project. The basic parameters of this
project are:
Number of competitive systems: n = 5

Global preference range: [Emin, Emax] = [77.35%, 94.89%]
Following are the final numeric results. For more detailed analysis please
see other LSP reports (SearchEngine.lst and SearchEngine.txt).
SORTED GLOBAL PREFERENCES
1. 94.89% Google
2. 89.78% bing
3. 88.18% Yahoo
4. 78.42% altavista
5. 77.35% Ask
SORTED NORMALIZED GLOBAL PREFERENCES
1. 100.00% Google
2. 94.62% bing
3. 92.93% Yahoo
4. 82.65% altavista
5. 81.51% Ask
Interpretation of results:
For the majority of standard criteria it is reasonable to request that

competitive systems satisfy at least 2/3 of requirements. If E<67% this
means either that the quality of the evaluated system is not sufficient,
or that the selected criterion specifies unrealistically high requirements
that cannot be sufficiently satisfied.
A reliability analysis is necessary to compute the level of confidence
that one system is better than all other systems. Small difference between
two global preferences (e.g. 3%) can be sufficiently significant if one
system outperforms others in the majority of elementary preferences.
An indicator of dominance of system A over system B can be defined as a
percentage of inputs where A is better than B.
Normalized results are convenient for relative comparison of analyzed
systems. Normalization is performed so that the maximum global preference,
the maximum global quality (Q), and the minimum global cost are interpreted
as 100%. Therefore, the preference of 85% is interpreted as 15% below the
best result. The normalized cost of 120% indicates the cost that is 20%
above the minimum cost. However, the relative comparison makes sense only
if absolute preferences are sufficiently high and absolute costs are in
the acceptable range.

Rajan Dhabalia LSP Research Paper

Diunggah oleh

Informasi Dokumen

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Rajan Dhabalia LSP Research Paper

Diunggah oleh

Hak Cipta:

Format Tersedia

Name: Rajan Dhabalia

LSP : Logic Score Preference

Here we discuss initial requirements to evaluate this model.

3.1 User Profile

3.2 Tools Used

1.1 Searching Input

1.2 Searching filter

1.3 Specific activity Search

Point additive discrete absolute criteria

[6.1] 1.1.1 Text Searching

Search with one keyword [Level of efficiency]

Search parts of - Search REL= 100 * S/ S max 60

Search part of statement [Level of efficiency]

Case sensitive Search [Yes=1 and No=0]

Ignore stop words - Ignore REL= 100 * Ignore/ Ignore max 80

Ignore stop words [Level of efficiency]

Numeric Expression Search

Compute numeric - Number of mathematical expression search engine can compute. 20

Compute numeric Exp [No of Expression]

Abbreviation Search [Level of efficiency]

Synonyms Search - SS REL= 100 * SS/ SS max 30

Synonyms Search [Level of efficiency]

Stemming Search - SS REL= 100 * SS/ SS max 20

Stemming Search [Level of efficiency]

Misspell Corrected - MS REL= 100 * MS/ MS max 30

Misspell Corrected Search [Level of efficiency]

Excluding Search - ES REL= 100 * ES/ ES max

Combinational - CS REL= 100 * CS/ CS max

Optional Search - OS REL= 100 * OS/ OS max

Optional Search [Level of efficiency]

Adjacent text search - ATS REL= 100 * ATS/ ATS max 50

Adjacent text Search [Level of efficiency]

Popularity based Search [Level of efficiency]

Content based - VCS REL= 100 * VAS/ VAS max

Content based Search [Level of efficiency] 50

Content Controlled - VCCS REL= 100 * VCCS/ VCCS max 25

Audio Searching [Level of efficiency] 30

[6.2] 1.2 Searching Filter

Security filter Weight

Pages with Spam,doorway [Level of efficiency]

Duplicate Content - DCS REL= 100 * DCS/ DCS max 45

Duplicate Content [Level of efficiency]

Adult Content filter [Level of efficiency] 90

Casino content filter [Level of efficiency]

Content Related Pages [Level of efficiency]

Pages related to same - DWS REL= 100 * DWS/ DWS max 35

Website related website [Level of efficiency]

Linked page - DLS REL= 100 * DLS/ DLS max 20

Linked Page [Level of efficiency]

RSS support pages [Level of efficiency]

Usage rights - US REL= 100 * US/ US max

Usage rights [Level of efficiency]

Numeric range filter - NS REL= 100 * NS/ NS max 35

Numeric range filter [Level of efficiency]

field to search keyword Search with specific field

Field to search keyword [No of Field]

Broken link filter - BLS REL= 100 * BLS/ BLS max

Page created time [Level of efficiency]

Recent update time - RUS REL= 100 * RUS/ RUS max

Recent update time [Level of efficiency] 80

Location of user [Level of efficiency]

Location of country - LCS REL= 100 * LCS/ LCS max 70

Location of Country [Level of efficiency]

[6.3] 1.3 Specific activity Search