Multimedia Systems

Characteristics of multimedia systems A multimedia system is an information system that combines different types of media; using at least 3 (2?) media types. They have the potential to provide a better experience than any other information medium. Types of media: Text Text refers to the letters, numbers and other characters. It has no meaning until a person reads and interprets it. Hypertext Hypertext allows the user to navigate through a multimedia product. A hyperlink is a highlighted item that allows the electronic connect. Hypertext provides the interactivity in a multimedia product. Audio Audio is sound that has been digitised. The meaning of the audio is determined by listening to and interpreting the sounds. Audio is used to explain concepts, reinforce selections and provide special effects. With the development of compression technology such as MPEG Layer-3 (MP3), transmission time, quality and storage requirements have greatly improved (lossy compression) Image The meaning of an image is determined by looking at it and interpreting it. All images on the screen are made out of pixels. A pixel (also known as picture element) is the smallest part of the screen that can be manipulated by the computer. The total amount of pixels that are on the screen is known as the resolution of the screen. There are two main types of images: Bit mapped graphics or vector graphics. Bit mapped graphics Bit mapped graphics (bmp) treat each pixel on the screen individually and represent this by bits in memory. This format is ideal for images that require shading and detail. When bit mapped graphics are transformed, they become ragged and suffer loss of resolution. Enlarging each pixel in a bmp creates a staircase pattern aliasing. They are often stored as GIF and JPG. Vector graphics Vector graphics are made up of objects, such as straight lines, curves or shapes. Common formats include CGM, EPS and WMF. Aspect Ratio The aspect ratio (constrained proportions) is the relationship between the vertical and horizontal size of an image. For example, a screen with a resolution of 1920 x 1080 pixels would have an aspect ratio of 16:9

Animation Animation is the movement of a graphic. It is the result of a series of still images, or frames, presented in rapid succession. Each frame that is used in an animation sequence is referred to as a cell. Video Video combines pictures and sounds displayed over time. Video starts with a continuous event and breaks it up into discrete frames. Print and multimedia The differences between print and multimedia (assuming similar content) are the mode of display, interactivity, ease of distribution and authority of document. Mode of display Printed information is displayed using a printer or plotter. On the other hand, a multimedia product is displayed using a screen and speakers. Screens display surface that provides immediate feedback about what the computer is doing. Speakers produce audio. Interactivity Interactivity allows the user to choose the sequence and content of the information. With printed information, this is not possible, as the user will have to go through the information to get to the desired information. However, in a multimedia system, tools such as hypertext (with particular reference to hyperlinks) make Interactivity possible. Hardware demands by multimedia systems

Frame Buffer The current image that is being displayed is stored in a section of memory called the frame buffer. High resolution images require more storage. The storage requirements of an image are dependent on the number, size, tone and colour of each graphic. Bit mapping The relationship between the image on the screen and the bits in memory is called bit mapping (also known as memory mapping). It is dependent on colour and depth. Bit depth Bit depth (Bit/colour/pixel depth) for images is the number of bits there are in a pixel. The bit depth of an image is used when calculating the file size of an image. File size of an image The formula for calculating file size is The Horizontal x Vertical x Bit Depth / 1024 x 8 bits (1kb). As an example, the file size (in kb) of an image containing 256 colours per pixel with a resolution of 1024 by 768 pixels would be: 1024 x 768 x 8 / 1024 x 8 = 768 kb

Bit Depth table

Bit depth

Number of colours or tones (each value doubles) 2 4 8 16 64 256 65 536 16 777 216 4 294 967 296

relationship

1 2 3 4 6 8 16 24 32

2^1 2^2 2^3 2^4 2^5 2^6 2^7 2^8 2^9

Colour Graphics Colour graphics and colour tones (grayscale) increase the number of bits per pixel, and thus require more storage. Colour graphics are obtained from an RGB screen that uses a combination of colours. The minimum amount is eight. Audio Storage Sound travels through the air in waves with a particular volume and pitch. A sound wave is analogue data. Audio is sound that has been digitised. It is created using an analogue to digital (ADC) converter and reversed by a digital to analogue (DAC) converter. The ADC and DAC are built into the computers motherboard or are added using a soundcard. Sampling The method used by an ADC to digitise a sound wave is called sampling. There are three important characteristics: Sampling rate The sampling rate is the number of times a slice is taken from the sound wave. 44.1 kHz and 22.05 kHz are two common sampling rates). A higher sampling rate means better sound quality and larger file size.

Sample size The sample size (bit resolution) is the number of bits there are in a sample. 192 kbps and 320 kbps are two common sample sizes, with the larger sample size having the better quality. Lossless audio (such as flac) can have sample sizes of over 1000kbps. Mono Mono uses one channel of sound, and stereo uses two channels (left and right) Size of an audio file An audio file size is calculated by multiplying the sample rate, sample size and the duration (in seconds). If the file is in stereo, the calculation is multiplied by 2. For example, the file size of a CD quality audio that uses a sample rate of 44.1 kHz with a 16bit sample size and a track that lasts for three minutes in stereo: File Size = (Sample rate x Sample size x time) x 2 = 44 100 x 16 x 180 x 2 bits = 254 016 000 bits = 31 752 000 bytes (1byte = 8bits) = 30.28Mb (1Mb = 1 048 576 bytes) Video Processing A video clip is used to create a video clip in analogue or digital form. Analogue signals need to be converted into digital using a video capture card. Software is used to edit the video May incl. adding text, audio or images. Video files are quire large. To satisfactorily display video or animations, the speed at which the image can be shown needs to be considered. Frame rate The speed of a video or animation is called the frame rate. It is measured by frames per second (fps). The high the frame rate, the smoother the video or animation. Full-screen video or animation requires a fast processer and a lot of RAM. Animation processing Animation creation is affected by the size of the image and speed of the movement. It is frequently set at 30fps, or 9000 frames for a five minute animation. With 30 frames being displayed every second, it creates a smooth illusion of movement. Drawing this many images is a large task fortunately, computer software makes this easier. There are two main ways of creating animation: cell based and path based animation. Cell based animation Cell based animation involves drawing and displaying individual frames or cells. Each frame is stored separately and loaded into a graphics page of primary memory. It is the traditional animation method. The animation works by displaying the first frame from a graphics page while creating the second frame in another graphics page. The second frame is slightly different from the first frame,

depending on the items to be animated. Animation occurs by alternating the display of graphics pages. A fast processor and a large fast-access storage device are required to ensure that the animation flows at the appropriate speed and so path based animation is preferred. Path based animation Path based animation involves displaying the movement of objects onto a fixed background in a particular sequence of path. The pixels in the background do not change, only the ones that pertain to the moving object. This method saves both memory and processing time. Tweening Tweening is when the object is drawn, wiped and then drawing the object is drawn in a new position. This process is made easier because animation software can generate the intermediate frames between two objects. Tweening gives the appearance that the transition from the first to second frame is smooth. Morphing Morphing is the smooth change between two different images. It transforms the shape/size/dimension/colour of one image into those of another. Warping Warping involves transforming or distorting a portion of the image. It divides the image into objects that can be manipulated.

People in multimedia systems Multimedia products are usually designed by a group of people who have an expertise in a particular field. There are people who are responsible for the system design, content and each of the media types, as well as people with information technology skills. Multimedia products require careful planning. People involved in multimedia systems are: Content providers These are the people who provide the raw material for the multimedia product (an unedited video, a series of images, etc.) System designers The system designer plans and organises the hardware and software required for the presentation. The system designer may also act as the project manager or provide technical support. Project manager The project manager organises the scheduling of various components of the product, provides overall supervision of the components and ensures that all components are delivered on time. Technical staff The technical staff includes a variety of artists and technicians who edit the graphic and video material, text and all other material into appropriate formats for the final document. Also includes layout and design personnel who work to develop and overall theme and look of the product. These people may also include the content providers, or may be a completely separate group.

Examples of multimedia systems

Education and training Multimedia systems are very effective at helping people to learn. A good MS uses different media types to maintain the users interest. The interactive nature of multimedia allows the user to control the learning. It addresses each users particular needs. They are a cost-effective method. Computer based training (CBT) type of multimedia product used for education And training. Person uses it at their own pace. Information is presented using different types of media. They are designed by experts and cater for different rates of learning The Typing Tutor.

Leisure and entertainment Generally computer games which provide a high level of interactivity. The responses of the user determine the game being played. Can also use some form of alternative to the keyboard joystick etc.

Information Multimedia systems provide users with an easy way to display information. The user controls when, how and what information will be displayed. Interactivity allows the user to search to find specific information advantage over printed forms. (Multimedia encyclopaedia) Information kiosks allow people to use a touch screen and select information about an organisation or service. Virtual reality MS are used for virtual reality and simulations. Virtual reality (VR) is the use of computer modelling and simulation to enable a person to interact with an artificial environment. VR is highly interactive. Flight Simulators Flight simulators are used to train pilots. Different scenarios are carefully modelled, and the reactions of the plane are made as realistic as possible.

Advances in technology Internet The internet has made it possible for information to be easily accessed multimedia to be presented without physically moving from your home computer. Speed has also contributed to Multimedia.

Broadband, ADSL, cable and satellite have significantly delivered great amounts of multimedia quickly; this has become beneficial for online news groups. Compression Thanks to the development of file compression, Files such as Mp3, QuickTime, Flash, Shockwave, Java, and Java Script have made downloading and online viewing faster than before. Too so it has allowed the popularization of music on the go with Mp3 players instantaneously storing and displaying music, without the hassle of storage consumption. CD-ROM On a CD-ROM, Each disk can store up to 700Mb, this effectively began the multimedia revolution and since been a popular method for storage. Driver speeds have also increased so more information may be accessed quicker from this storage medium. DVD/Blu-ray DVD is a latter technology rapidly taking over CD. The significance of DVD has become monumental, with the ability to burn on either side hence doubling the capacity of information held (17Gb). The popularity of DVD storage can be seen with its phase out of VHS as the method for movie storage. DVDs competitor, Blu-ray, a faster and greater storage medium than DVD has become commercialized. Its apparent higher quality has made it a selling point for movie distributors, though the difference to DVD is minimal, the only change being recognised by High Definition, high resolution television. USB The USB (Universal Serial Bus) has become a popular more convenient storage method has conquered the slower CD. With storage capacity of almost 20Gb, and Mp3 players which utilise the same hardware containing almost 200Gb; instantaneous computer access to information make this a popular multimedia storage medium. Creating and displaying multimedia Display hardware Cathode Ray Tube
o o o o o

The most commonly used VDU is the CRT Images are produced on the screen when an electron beam strikes a phosphorescent surface The phosphorous glows briefly when the beam hits, so the beam must always be on so the image can constantly be reproduced The image is continually reproduced by the beam going left to right from top to bottom of the screen. This pattern is called a raster scan. The time to complete a raster scan is called the refresh rate, and ranges between 60-100 scans per second.

Touch screen
o o

Is a VDU sensitive to human touch, allowing the user to interact with the computer by touching the pictures or words on the screen. Resistive the screen panel is coated with a thin metallic, electrically conductive and resistive layer. When the panel is touched it causes a change in the electrical current which registers as a touch and is sent to the controller for processing the response. Surface wave uses ultrasonic waves that pass over the touch screen panel. When the panel is touched, a portion of the wave is absorbed. This change in the ultrasonic waves registers the position of the touch and sends this info. to the controller for processing. Capacitive this is coated with a material that stores electrical charges. When the panel is touched, a small amount of charge is drawn to the point of contact. Circuits located at each corner of the panel measure the charge and send the info. to the controller for processing.

Liquid Crystal Display

o o o o o o o

LCDs are common VDUs in Notebooks and laptops, and have since been used a smaller alternative to the robust CRT screens. The heart of the liquid crystal display is a piece of liquid crystal material between a pair of transparent electrodes. The liquid crystal changes the phase of the light passing through it and can be controlled by a voltage applied between the electrodes. Hence the rows and columns of the screens electrodes light up. LCDs have become a popular form for advertising. LCDs use less power than CRT and emit smaller levels of electromagnetic field radiation than CRT. LCDs are volatile, if touched they can be easily damaged, causing a black spot as the transistor has been damaged.

Plasma
o o o o

Plasma creates red, green, and blue lights in every pixel, reducing the need for space. Charged electrodes between two glass panels react with tiny particles of inert gas. The gas changes to state of plasma, resulting in UV light, which then reacts with RGB phosphor in each pixel to emit visible light. The image compared to dull LCD makes it sharper and brighter. And unlike CRTs or LCDs needs not be located far away from peripheral speakers as magnetic fields dont affect it. However its processing ability is slower than LCD and has difficulty focusing on fast moving objects, often having a trailing effect. Plasma requires a lot of electricity run consequently producing mass heat, and has a limited lifespan of 30,000 hours.

Video projectors
o

These were once extremely expensive and only worked in dark rooms, however one thing has changed, that is the display no longer require a dark room and picture quality is much higher

Three types of projector exist: Three guns: Each gun for R,G, and B colours, these may produce the most light output but they are highly expensive. DLP: uses a tiny multifaceted mirror chip to reflect light from a light source. Each mirror is controlled by supplying a TV signal; the chip is then able to create an image which is projected onto the screen. LCD-based projectors: perform similarly to LCD screens. Light is passed through and LCD before being projected onto a screen.

Speakers and sound system

o o

Sound has become a vital part of multimedia use. It is the sound card that determines the quality, not the speakers. The purpose of the speakers are simply to amplify the sound wave, although good quality speakers will lead to better quality sound.

Head sets and Heads up displays

o o

VR often use a headset to impose the perception of reality directly to their visual senses. Two VDUs are used, each constituting one side of the glasses, and each presenting a slightly altered view than the other. When the two are combined a 3D image is presented. Heads up displays are smaller, less intrusive, and were developed by the military. The screen is transparent so forward view is not affected.

Display software Most software used for multimedia is WYSIWYG: what you see is what you get, it shows at all stages of production what the result will look like. Authoring software allows users to create multimedia products using both print and multimedia, recording the final product onto CD, DVD, Tape, Hard Disk Drive or other electronic media. Presentation software allows users to make automated slide presentations. A slide may contain text, audio, images, animation or video, and it is easy to create links between slides (hypertext). Digital video editing package Allows the user to create transitions, titles, and sound, and to record sound and video clips.

Web browsers and editors assist users in creating and editing multimedia material on the Internet for locating and viewing.

Collecting and organising processes in multimedia systems Images Images collected for an information system must be digitised. Digitising involves converting an image into bit patterns within a computers memory. This can be achieved by scanning an image into the computer, or uploading it via digital camera .An optical scanner shines an intense beam of light at a paper document. Light detectors measure the intensities and colours of the reflected light. Analogue to digital converters then turn these measurements into digital bit patterns that can be stored for use later. Size and quality of the image are affected by the DPI (dots per inch), which represents the number of dots that will be created for every inch of the scanned image, essentially the controller of resolution. Video A video digitiser measures and records signals that represent colours and light intensities .The PAL video system used in Australia contains 625 horizontal scan lines. Each line contains data that represent the colour and intensity of every pixel in that line. Each line is refreshed 25 times per second. Digitising an analogue signal requires special hardware (set top box) and a very fast system. Audio Sound is digitised by taking regular measurements, during each measurement the amplitude (volume) of the sound is measured and converted into a value. The number of samples taken every second is called the sampling rate. The most accurately digitised sounds will be created by using the highest possible sampling rate. However the higher the sampling rate, the more data samples exist and hence more space is required. Bit resolution Sound samples can be stored either as 8-bit or 16-bit binary number. 16-bit is clearly the higher quality sound but requires twice the storage. Organising Storyboard It is a scene-by-scene layout of the multimedia presentation. There are three (sort of four) types of storyboard layout: Linear Layout is sequential

Hierarchical Choices branch off into further choices, based on navigation Non-linear Allows total freedom in navigation, though presentation is confusing Composite A mixture of all layouts

Storing and retrieving/transmitting and receiving processes in multimedia systems File formats Audio

WAV MIDI MP3

Video

AVI MPEG QUICKTIME

Images

JPG/JPEG EPS PICT TIFF BMP GIF

Text/numbers

TXT DOC PDF HTML RTF

Compression and Decompression Video Compression Achieved by the use of a codec, an acronym for compression/decompression algorithm. Examples include MPEG-1, -2, and -3, created by the Moving Pictures Expert Group. Image compression Minimizing the size in bytes of a file without degrading the quality. This is a popular choice when images are used over the internet. Compression methods: Lossless Decompressed data is identical to the original data with nothing lost in translation, e.g. GIF compresses by replacing patterns with symbols. Lossy Where the decompressed data is slightly different from the original, some data is however sacrificed for compression, e.g. Mp3, Jpeg. Decompression Can only be achieved if the same algorithm containing the original compression information Transmitting and receiving Data streaming allows multimedia data files (video and audio) to be decompressed and played as they arrive. There is however a number of issues, including transferred data packets going missing there isnt enough time to ask and send the data again. Networks running at different speeds (Dialup/broadband). Buffering prevents said problems via prediction, compression/decompression, and error correction methods.

Issues related to multimedia systems Copyright Content Merging There is a confluence of technology merging with the Internet, e.g. TV, radio, with the advent of live streaming. Data Integrity Data is valid and verified.