0" in the
Context of Design Automation
[Aurangabad–
Dr. T V09Gopal
October 2010]
Professor &
Chairman, CSI Division II [Software] &
Honorary Chief Editor - CSIC
Department of Computer Science and Engineering
College of Engineering
Anna University, Chennai – 600 025
e-mail : gopal@annauniv.edu
Home Page http://www.annauniv.edu/staff/gopal
Cobb’s Paradox:
"We know why projects fail, we know how to prevent their failure – so why
do they still fail?"
“The tar pit of software engineering will continue to be sticky for a long time to
come. One can expect the human race to continue attempting systems just
within or just beyond our reach; and software systems are perhaps the most
intricate and complex of man’s handiworks”
– F.P. Brooks
The Crux
"On a large software project one is lucky if
one person in 50 has anything resembling
an overall understanding of the conceptual
structure of the project, and divinely
blessed if that person has the ability to
explain it in lay terms"
- G. Robinson
"The outcome of any software project is necessarily uncertain…There
is no problem ‘producing’ software – the problem is knowing what to
produce" - A. Hall
Winning with Software
“While technology can change quickly, getting your people to change takes a great deal longer. That is why the
people intensive job of developing software has had essentially the same problems for over 40 years. It is also
why, unless you do something, the situation won’t improve by itself. In fact, current trends suggest that your
future products will use more software and be more complex than those of today. This means that more of your
people will work on software and that their work will be harder to track and more difficult to manage. Unless you
make some changes in the way your software work is done, your current problems will likely get much worse.”
- Watts Humphrey, Winning with Software: An Executive Strategy, 2001
Rapidly Changing Global
Risk Landscape
Unimagined new sources of risk
Unimagined risk correlations
Unimagined emergent vulnerabilities
Traditional Models
Aimed at business
Contextual
executives
Logical
Aimed at architects
and designers
Physical
Aimed at designers
and developers
Old Model – Microsoft Solutions
Framework [MSF 3.0 + Views]
Business strategies &
Contextual processes
Applications View
Technology View
Information View
Applications to facilitate
Business View
Information needed to
Logical manage business
Technology to support
Physical business & application
needs
The Need of Architecture
The Winchester “Mystery” House
Predicting the Dynamics
Key Challenges
1. The levels of professionalism observed in software engineering are
generally lower than those in other branches of engineering, although
there are exceptions.
2. Education in many universities is not producing IT practitioners with the
IT application and project skills they need.
3. The importance of project management is not well understood and
usually under-rated and senior managers are often ill qualified to
handle issues relating to complex IT projects.
4. Risk management is critical to success in complex projects but is
seldom applied effectively in the case of IT and software.
5. The vital role of the systems architect in major IT projects is frequently
not appreciated and there is a shortage of appropriately skilled
individuals.
6. There is an urgent need to promote the adoption of best practice
amongst IT practitioners and their customers.
7. Basic research into complexity and associated issues is required to
enable the effective development of complex, globally distributed
systems.
1. Complexity Science
Software 2.0 – Emerging 2. Systems Thinking
Competencies 3. Cybernetics
4. Web Intelligence
5. Software Aesthetics
Understanding Complexity
Examples of complex adaptive systems include:
The patterns of birds in flight
The interactions of varied life forms in an ecosystem
The behavior of consumers in a retail environment
The rise and fall of species in evolution
The interactions of people and groups in a community
And other complex social, biological and ecological phenomena.
The findings of Complexity science can be used for more natural, more
productive, more enjoyable and far more innovative results managing people,
organizations and communities.
Old Science and Reality Checks
Things happen because something causes them to
happen (cause and effect).
We can understand what happened by reducing things to
their components or parts and examining those parts
(reductionism).
The universe is orderly, follows natural laws, and works
like an incredibly complicated machine.
The best way to manage people is to organize them into
a clear structure and control them with clear directions.
The best results occur when work is streamlined to be as
efficient as possible, with a minimum of wasted effort,
producing the most output in the least amount of time
(the "lean machine").
Chaos – Making a New Science
The new science of Complexity evolved
out of general systems theory and a field
of study known as chaos theory.
The New York Times reporter popularized
such insights as the Butterfly Effect,
whereby a butterfly flapping its wings in
India causes a series of air movements
that eventually result in a thunderstorm
over Chicago.
Complexity Science
Then in the 1990s a group of brilliant scientists (including several
Nobel winners) affiliated with the Santa Fe Institute in New Mexico
said, in effect, "There's not much point in studying chaos. It's too
chaotic. Let's study Complexity, where with the help of computers
we can actually figure something out."
They have discovered many things about "the real world" with
practical applications for business, management, community and
economic development.
Controlled System
Controller
control manipulated
control Actuator
input variable
function
error
sample controlled
+ Monitor
- variable
reference
Closed-Loop Controller
The output of the system y(t) is fed back
through a sensor measurement F to the
reference value r(t).
The controller C then takes the error e
(difference) between the reference and the
output to change the inputs u to the
system under control P.
Advantages of Closed-Loop
Controllers
Disturbance rejection
Guaranteed performance even with model
uncertainties (when the model structure does not
match perfectly the real process and the model
parameters are not exact)
Unstable processes can be stabilized
Reduced sensitivity to parameter variations
Improved reference tracking performance
Open-loop control
The view of A realist view of epistemology: A biological view of epistemology: A pragmatic view of
epistemology knowledge is a “picture” of how the brain functions epistemology: knowledge
reality is constructed to achieve
human purposes
A key distinction Reality vs. scientific theories Realism vs. Constructivism The biology of cognition vs. the
observer as a social
participant
The puzzle to be Construct theories which Include the observer within the Explain the relationship
solved explain observed domain of science between the natural and
phenomena the social sciences
What must be How the world works How an individual constructs a How people create, maintain,
explained “reality” and change social
systems through
language and ideas
A key assumption Natural processes can be Ideas about knowledge should be Ideas are accepted if they
explained by scientific rooted in neurophysiology. serve the observer’s
theories purposes as a social
participant
r0 observed
remaining errors
schedule set by
the manager
rf
cp1 cp2 cp3 cp4 cp5 cp6 cp7 cp8 cp9 t- time
t0 cpi = check point i
deadline
Physical and Software Systems: An
Analogy
External Dashpot
force
To err is
Human Block Rigid surface
Spring
First Law:
Third Law:
Guest Editor:
Dr. Raghu
Ramakrishnan,
Chief Scientist for
Audience and Cloud
Computing,
Yahoo!
Web Intelligence
"The information superhighway is a dirt road that won't be
paved over until 2025.“
- Sumner Redstone, CEO of Viacom/Blockbuster
- Web Intelligence includes domain expertise in Social
Networks and Social Intelligence, Web Mining and
Farming, Semantics and Ontology Engineering, Web
Information Filtering and Retrieval, Web 3.0, Web Agents
and Agent-based Systems and Knowledge Grid and Grid
intelligence.
- How all these strategies relate to the way Human Brain
process information and operates on it is a more
challenging question.
As We May Think
Article by Dr. Vannevar Bush in The
Atlantic, July 1945
the idea of hypertext or "instant cross-
referencing," decades before the term
itself was conceived by Ted Nelson and
before the concept was widely
implemented on the World Wide Web.
Bush called it "memex" device
WWW – At a Glance
Current Trends
Digital Libraries
Hypermedia [Entertainment]
Agent Technologies
E-Commerce
E-Learning
E-Governance
Multi-Tiered Architectures
Database Connectivity Methodologies
Web Db Functions
Semi-Structured Data
RDBMS – Fully structured data ; each
element has a well defined type
Multimedia data is unstructured
[Impossible to represent]
To better represent such data, one
introduces some (partial) structure.
Such data are called semi-structured data
Web Services
Web Services address the need for interoperability
between web applications by specifying a standard
XML based protocol.
Please recall the hassles with CORBA, DCOM and
the likes.
One of the better-known early implementations of
XML-based communication protocols is XML–RPC
(XML Remote Procedure Call).
Most vendors did not fully embrace it, but rather
chose to embrace its successor SOAP (Simple
Object Access Protocol)
Web Services Layers
WSDL : Web
Service
Description
Language
Web Services
Can be secured using standard strategies
adapted to XML
Can be adapted to GRID
Can be adapted to ‘semantic web’
Appear to be the best bet for EAI
Can be dynamically composed [There are
composing tools]
Technical Perspectives of
‘The Web’
Computer science perspective - infrastructure
and intelligent systems
Information science and knowledge
management perspectives - data, information,
knowledge, wisdom hierarchy
Social intelligence perspectives - connectivity,
social network intelligence
Application perspectives -E-commerce, E-
Governance, E-Learning and so on.
Perspectives of ‘Science’
Physical/biological science perspectives -analytic
disciplines that aim to find laws/processes that generate
or explain observed phenomena
Social science perspective– scholarly or scientific
disciplines that deal with the study of human society
and of individual relationships in and to society
Computer science perspective - a basically synthetic
discipline that creates mechanisms (e.g., formalisms,
algorithms, etc.) in order to support particular desired
behavior
Web Science
The Web is a new technical and social
phenomenon and a growing organism
The Web needs to be studied insitu and
understood and it needs to be engineered
Web Science is a new field of science
that involves a multi-disciplinary study and
inquiry for the understanding of the Web
and its relationships to us
Intersection of Disciplines
Roots of Web Science
Web Ecology (Bernardo Huberman) – “The Web becomes a
gigantic informational ecosystem that can be used to
quantitatively measure and test theories of human behavior and
social interaction.” (The Laws of the Web, 2001)
Web Engineering – “…covers the realization of solutions within
the Web, its applications and its advancement, in particular its
approaches, methods, models, principles and tools, which are
based on the information and communication technologies of the
Internet” (ISWE)
Challenges
Web Science suggests that
We can use non-technical concepts to understand
the complexity of our Web applications so that we
can engineer them to have new and predictable
behaviors
We can better understand the impact of Web
technology in all areas of communication and
social interaction
We may be able to reliably predict the future
evolution of the Web
Software Agents
An intelligent agent is a piece of software that
performs a given task using information from its
environment and acts in such a way that it can
complete the given task successfully.
Some desirable properties for such agents are:
autonomy, adaptability, mobility, reactive,
pro-active, social and communication ability.
To deal with complex real world problems, it is
desirable to have different type of agents.
Key Challenges
1. Semantic Heterogeneity: Structured
and Unstructured data
2. Support for Dynamic Evolution of
Information Sources
3. Scalability and Performance
4. Generalization across application
domains
Information Architecture
Approach
Certifies the authenticity, cost and degree of
completeness of the content.
Establishes and documents the context (who,
what, why, where and how aspects that are
extrinsic) of a content. The special context of
preserved and archived collection if any is also
established.
Identifies and exploits structural relationships that
exist between and within a document.
Provides a range of intellectual access points for
an increasingly diverse range of users.
Information Architecture
Approach
Provides some of the information a professional
might have provided in a physical reference or
research setting.
Facilitates multi-versioning of the document (various
forms of the same document).
Tracks acquisition, stocking and accessioning
information over the entire network.
Note : Document = Information Object
Semantic Web
"The first step is putting data on the Web in a
form that machines can naturally understand,
or converting it to that form. This creates what
I call a Semantic Web—a web of data that can
be processed directly or indirectly by
machines."
--Tim Berners-Lee, Weaving the Web, Harper
San Francisco, 1999
Key Reasons
1. Information Overload
2. Stovepipe Systems : No sharing of data
3. Poor Content Aggregation
Smart data Continuum
The Stages of Smart Data
Text and databases : The Applications are
smart
XML documents for a single domain
Taxonomies and documents with mixed
vocabularies [Classification]
Ontologies and rules [Inference of new data ;
facilitate knnowledge sharing and reuse]
Semantic Web: a machine-processable web of
smart data.
Web Services
Semantic Web Vision
RDF : resource Description Format
Software Aesthetics
“Beautiful Code: Leading Programmers Explain How
They Think”, Andy Oram and Greg Wilson [Editors],
O'Reilly and Associates, Sebastopol, CA, USA, 2007.
“33 essays by eminent programmers on the art and craft of
programming”
“For both essays and computer code, it’s always important to look at how each
one is written. Even if the idea itself is good, it will be difficult to transmit to the
desired audience if it is difficult to understand. The style in which they are
written is just as important as their purpose.”
- Yukijiro Matsumoto, Chief Designer of the Ruby Programming Language,
Beautiful Code, O’Reilly, 2007
Some Case Studies
NCSL : Non-Comment Source Lines
Growth in functionality provided by software to pilots of military aircraft
Systems that must manage complex interactions and high coupling are more
prone to accidents. Space missions are among these high-risk systems.
US DoD - Software Engineering and System
Assurance
Deputy Directorate Laments [2007]
Software requirements not well defined, traceable,
testable.
Immature architectures, COTS integration,
interoperability.
Software development processes not institutionalized,
planning documents missing or incomplete, reuse
strategies inconsistent.
Software test/evaluation lacking rigor and breadth.
Unrealistic schedules (compressed, overlapping).
Lessons learned not incorporated into successive builds.
Software risks/metrics not well defined, managed.
Standish Group Top 10 Factors Most Responsible for Project
Success or Failure
Complexity and Success / Failure
Factors
1. Complexity Science
Software 2.0 – Emerging 2. Systems Thinking
Competencies 3. Cybernetics
4. Web Intelligence
5. Software Aesthetics