ITEC2110:Fall2010:Section12:Desiree’ Delgado:Class Notes

From GGCWiki
Jump to: navigation, search

ITEC2110:Fall2010:Section12:Desiree’ Delgado:Class Notes

Contents

Return to My Digital Media Home Page

Class Notes from ITEC 2110-12 Fall 2010

August 31, 2010

No notes. Notes will be begin September 2, 2010.

September 2, 2010

Multimedia can be divided into time-based or static media. It can also be linearly or non-linearly structured.

Time-based means that it occurs over a period of time. Static media occurs at a certain point of time.
Linear means that it occurs from beginning to end in an accepted order. Non-Linear means that people have a choice or that it can go backward, forwards, or skipping around.

Looked at the pros and cons of the advancement of technology.

It takes time for conventions concerning usage to be formed. At first the new form tries to act like the old form to make new stuff look more familiar.
Example: The history of film: no sound to performed sound to recorded sound.

We will be focusing on media forms transmitted across the internet and how to manipulate them, including: audio, still images (bitmapped and vector graphics), moving images, and animation.

Wiki page is not a project yet, it is just the base for what we will be doing in the future. We shouldn't have any audio or video on there yet.

Interactivity:

User input may control a multimedia production, but only within limits set by the multimedia producer. And only choices that are coded into the program are possible.
Can allow the user to control events at many points, leading to combinatorial growth in a number of possibilities. This means that if there are 4 choices at 4 different points then there are 1024 (4 to the 5th power) possible sequences that could happen.

User Interface:

Standard Forms: buttons, sliders, radio buttons, check boxes, and pull-down menus.
Check Boxes are different from radio buttons because check boxes are typically used for multiple choices.
Sliders are when you have a button and you slide it left and right or up and down to select the level that you want.
Pull-down Menus are typically used in order to save space on the page.
Custom Forms: design, build, and publish your own.

Ethical Considerations:

Is technology ethical or unethical.
New technology affords new means of interaction and opportunities for behavior that previously didn't exist.

Social Considerations: Access

Technology is expensive.
1 in 100 people worldwide have a computer.
Go to http://www.snopes.com/science/stats/populate.asp to view the world if it were reduced to 100 people.

Why such a small amount of people?

If access to multimedia is the norm, those denied access become marginalized ("digital divide").
Access may be limited by lack of: access to equipment/skills, network infrastructure, literacy/education, and physical/cognitive abilities.

WWW use broadens:

Initially seen as a source of information. Seen as only economic in nature.
Now it is also a transmission medium: streaming content (live radio), streaming content (tv shows online), and podcasts.

This pretty much sums up the first chapter. Next topic is on the number systems and how these systems run, part of Chapter 2. Read the slide on the website.

September 7, 2010

No notes.

September 9, 2010

No notes.

September 14, 2010

Continuous has to be measured. The depth of an ocean, the height of a wave, etc.

Continuous phenomenon to digital data: digitization requires two processes: sampling (equally spaced) and quantization (measuring at each sample).
Digital data back to continuous phenomenon: display samples using "sample and hold". Play the sample for the duration of the sample time. The collected sample will be used to recreate what happened earlier as the event will be over.

Sampling and Quantization

Example: Every hour you get a sample and its value. Then you record the information, in quantization there can be no partial number so any decimals must be rounded to the closest whole number. This information can then be used to recreate what happened during the period of time when the data was collected. It will not be exactly like the original curve because of the rounding, so this will not be as accurate but will provide a rough idea.
Sampling Rate - the number of samples collected in a fixed amount of time or space.
Quantization Levels - the set of values to which a signal is quantized. This is the measurement used and then mapped to quantization levels.

What Sampling Rate to Use?

Sine Function - a function that repeats itself after each cycle. Cycle - the distance between where two points reach the same place.
Single Sample - typically taken from the middle of the measurements. This is typically not a good way of sampling because most of the data is lost and does not represent all of the data available. This is very inaccurate.
Two Samples - typically taken at 1/3 and 2/3 of the time. This is a little bit better in recreating the original curve. Still a lot of data is missing and typically is very inaccurate and can be insufficient for collecting data.
Three Samples (Sample and Hold) or more - the more samples taken, the more accurate the quantization will appear and the more it will appear like the original curve.
Nyquist Rate = 2fh (two times the highest frequency). This formula will give the sampling rate that should be used.
Example - Human hearing response is in the range of 20 to 22,000 cycles per second. So Sampling Rate = 2(22,000) = 44,000 samples per second. This would be a good quality CD to be recorded at.

September 16, 2010

No Notes.

September 21, 2010

Vector Graphics

Elegant way to construct digital images that have a compact representation, are scalable, are resolution independent, and are easy to edit.
Mandatory for 3-D modeling and is impractical using pixels.

Coordinate Systems

Bitmapped pixel coordinates (integer - whole number only) this is the (x,y) coordinates that are used when graphing to represent points. This can be used to specify a point on a monitor. Top left corner is (0,0) and x increases as you go right and y increases as you go down.
Vector graphics coordinates (real values - actual values, can be decimals). A point is defined by its x and y values.
Displacement - distance between points or movement from one point to another. This can be defined by a pair of points.
Example: Point 1 (1.41, 1.74) and Point 2 (3.12, 0.95). To figure out the displacement it would be x = 3.12 - 1.41 = 1.71 and y = 0.95 - 1.74 = -0.79 so the displacement would be (1.71, -0.79). Vectors have magnitude (length) and direction. The length in the example above is -0.79 and the direction is 1.71.

Vector Graphics

A grandfathered term
From the time when the position of electron beam of CRT was controlled by setting coordinates of images.
Give the display two points and it would move the beam from one point to the other drawing a line.
This movement could be described as a vector since it has magnitude and direction.
Stores shapes very economically in the form of formulas and equations.
A line is defined by - y=mx + b which is an infinitely long line.
In computer graphics we want a line segment with end points.
A line segment can be completely described by its endpoints.

Vector Graphics: Curve

Bezier Curves - are smooth curves that can be set by two end points. A cubic Bezier curve has 4 control points: 2 end points and 2 direction points. The sweep of a Bezier curve is determined by 1 less than the number of control points.

Quadratic Bezier curves only have a single direction point and they are the only Bezier curves supported by SWF. PDF and SVG provide both cubic and quadratic Bezier curves.
SWF = Short Wave File
PDF = Portable Document File
SVG = Scalable Vector Graphics

Line and Curves

Bezier curves can be smoothly joined together.
An anchor point is the point where one joins the other.
Closed lines can be filled
Fill can be filled with solid color, pattern, or gradient color(s).
Lines have ends that can be round, square, etc.

Manipulating Objects

Translation is a simple up/down and/or side-to-side movement.
Scaling: make bigger or smaller.
Rotation is sliding an object around a certain point.
Reflection is flipping the image around a certain line.

September 23, 2010

Question: How many different colors can be created using an RGB color scheme that has a 24 bit color depth.

Answer: 2^24 = 16777216 different colors

Ct'd Chapter 3

Object Fills

Solid Color
Patterns
Gradients - special tool for this in Inkscape.
Linear
Radial
Fill is the color inside the shape.
Stroke is the boundary line around the shape.

Note: Difference between bitmap graphics and vector graphics: bitmap uses pixels and vectors are more scalable and easier to edit.

3D Graphics

Three axes - x, y, and z. This gives us the ability to zoom in and zoom out. In 2D we do not typically have that ability.
Much more complicated than 2D. 3D gives us spheres, cubes, and planes.
3D shapes are defined by their surfaces and leads to more complicated math.
3D is even more complicated on computer because the 3D object in the computer must be translated to a 2D to be rendered on a computer screen. This results in the need to specify the viewpoint.

3D Additional Complexity

Lighting - position of light sources, intensity, color, etc...
Surface Texture - color, texture, interaction with lighting sources, etc...
Rendering is extremely computationally expensive (demanding). Just 10 seconds of animation can require up to an hour to render it.

3D Modeling

There are 3 general approaches to modeling in 3D:
Constructive Solid Geometry - uses geometric solids (cube, cylinder, sphere, pyramid), objects built by squishing and stretching those geometric solids, and can be built by joining objects using union, intersection, and difference.
Union is when two objects occupy the same space.
Intersection is only the area where two or more objects overlap.
Difference is everything except where the objects overlap.
See http://en.wikipedia.org/wiki/Constructive_solid_geometry for examples and more information.

3D Models - Free Form

Uses an object's surface to define it.
Build surfaces from flat polygons or curved patches. Flat polygons are easier to render and, therefore, frequently used in games where computational power is limited.
Results in an object drawn as "mesh".

3D Model Free Form - Extrusion

Draw a 2D shape and then give the shape a depth and it will create a 3D shape.
Lathing - uses a curve to create a complex shape. The curve is then rotated 360 degrees using a vertical line that will result in a 3D shape.

3D Models Procedural Modeling

Objects are defined by formulas
Best know is based on Fractals.
See http://en.wikipedia.org/wiki/Fractals for more information and images.
See http://en.wikipedia.org/wiki/Sierpinski_gasket for another example of constructing fractals.

3D Rendering

Rendering engine handles complexity
Wire frames are used to preview objects and their position - can't tell which surface is closer to us and which surface is hidden.
To save computation time, hidden surfaces are removed before rendering.
Lighting - added to scene much like an object, type of lighting, position, and intensity.
Direct relationship between rendering quality and computational burden. The higher the quality, the higher the burden.
Shading - how light reacts with surface. Based roughly on physics but modified by heuristics.
Texture mapping - an image is mathematically wrapped around the object.
Light reflecting off objects of one color affect the color and lighting of surrounding objects.
Two Methods:
Ray Tracing Complex - must be repeated for pixel in the image. Photo-realistic results.
Radiosity

NOTE: There will be a take home quiz available online later today (word doc), bring it to the next class.

September 28, 2010 TEST REVIEW

Test Review:

25 Questions on Test: Couple Multiple Choice and the rest are short answers. Can have calculator.

Media - what is media? digital media? multimedia?
Media - way of communicating information in multiple formats.
Digital Media - same as media in a digital format.
Multimedia - more than one media method at a time.
Static Media - happens at a specific period of time.
Time Based Media - happens over a period of time or space.
Interactivity of Media
Number Systems
4 Number Systems and their digits
Know how to convert between binary and decimal numbers
Know how many different things can be represented given x number of digits (2^x).
Digitization
Discrete - Can be counted.
Continuous - has to be measured.
Sampling - take data at equally spaced intervals.
Over-Sampling - taking too many samples. Good quality but large file size.
Under-Sampling - taking too few samples. Bad quality but small file size.
Quantization - map a sample to a quantization level.
Quantization Levels
Posterization
Be able to calculate an audio's file size from the sample rate, depth, and play duration.
Know continuous and discrete phenomena (sampling vs counting)
Retrograde Motion - too few samples without a reference point that make something appear to be going backwards even when it is really going forward.
Know difference between bitmap graphics and vector graphics and conversions between the two.
Conversions Between the Two - going from Vector to Bitmap is easy, but the reverse is hard.
Know the concept of rendering - a process of grabbing the binary digits from the hard drive and displaying them on the monitor.
Know typical software used for vector graphic editing - InkScape, Autocad, Adobe Illustrator, and Adobe Flash.
Know vectors and vector addition and subtraction.
Ex. Given P1 (4,3) and P2 (8,6) find P3. (8,6)-(4,3) = (4,3)
Know common shapes in vector graphics:
Rectangle and Squares
Circles and Ellipses
Lines and Curves
Know about the Bezier Curve - has a bunch of control points that define the curve. These points are classified as end points and direction points. The degree of the curve is always 1 less than the total number of control points.
End Points - where the curve starts and end.
Direction Point - control points that manipulate the shape of the curve.
Know basic editing operations on vector graphics objects:
Scaling - make bigger or smaller.
Translation - move around.
Rotation - spinning around one point.
Reflection - flipping over a line.
Sharing - skewing an image.
Know about 3D graphics using vector graphics
General 3D modeling approaches and techniques used in each
Constructive Solid Geometry - uses union and difference.
Free Form Modeling
Procedural Modeling
Important components in 3D modeling
Lighting
Texture Modeling
Etc

Quiz Review

Should have all 3 quizzes graded.

Quiz 1 answers
1 Which of the following is time-based media? Audio and Movie
3 Which of the following is non-linearly structured? WWW (the internet) and Flash Game.
5 What is correct syntax to display an image on a your wiki page? [ [image:filename] ]
Quiz 2 answers
1 How many possible different binary numbers can be listed if it is 5 digits long? 2^5 = 32
2 The number that comes after 79 in hexadecimal number system is 7A.
3 1010 must be a binary number since it contains only 1's and 0's. False it can be but does not have to be.
Quiz 3 answers
1 Retrograde Motion happens because of too few samples.
2 How many bits are in one kilobyte? 8,000 bits
The rest of the answers are under submissions.

Additional Review Guidelines

Review the slides
Textbook is not a major resource.
Quizes - make sure you know the answers.
Wiki Book Notes and Questions - look at the information that previous students have asked.
Test Study Questions from previous students.
Sample Question 1:
How big is an audio file that is 44,100 samples per second with a 16 bit quantization and is 3 minutes 16 seconds in length in stereo?
Convert time to seconds 3 minutes x 60 seconds = 180 seconds + 16 seconds = 196 seconds total
Convert quantization to bytes 16/8 = 2 bytes
Stereo indicates the number of channels (2). If mono-stereo means 1 channel. Use the number that applies to the question.
Then multiple 44,100 samp/sec x 196 sec x 2 bytes x 2 channels =
Sample Question 2:
How big is the audio file given: 1574 pixels wide, 1987 pixels tall, RGB encoded, 8-8-8 bits each, no compression, no table.
Convert bits to bytes. 8*3 = 24 /3 = 3 bytes
1574 x 1987 x 3 = 9,382,614

Note: In black and white text, you will see a lot of zeros because of the black text.

Review slides will be available on blackboard.

October 5, 2010

Overview of Test

Class Average: 66-67
No curve; however, if your final exam is higher than your lowest test grade, then the final exam will replace your lowest test grade.
Two lowest quiz grades will be dropped.
Next test, can have a 4x6 index card as a cheat sheet.

Chapter 4 - Bitmap Graphics

Device Resolution - how many pixels are used to display an image.
Determines how finely the device approximates the continuous phenomenon. Type of device used, type of monitor, etc has an effect on how well the image is shown.
Is closely related to sampling.
Can be expressed in a number of different ways:
Printers and Scanners: dpi (dots per inch)
Video: number of pixels, pixel dimensions (640x480 or 768 x 576 or Webcam 320 x 160)
Compression of Images - lossy and loss-loss compressions.
Image Manipulation - goes with our project.
Convolution Masks - tells what actually happens when we use special effects.

See slide show for more details.

October 7, 2010

Data Compression

Pixels x Bites = total number of units needed to store the image.
RLE Compression - follow color left to right until the color changes. The last pixel on the right wraps to the first color on left on the next row, if the color is the same, then the coloring is still going.
x number (one byte per number) + y RGBs * 3 bytes = number of bytes for storage.
Pro - This is best to use when there a small amounts of changes in the color. Best case is if it is all in one color.
Con - Worst case is if the pixels change color one after another (this would be ineffective and could lead to a larger file size.
RLE is LossLoss

Dictionary-based Compression Technique

Construct a table of strings (colors) found in the file to be compressed.
Each occurrence in the file of a string (color) found in the table is replaced by a pointer to that occurrence.
This is similar to a children's book. You may have a book that is 2,000 pages long, but only uses 2,000 words. The same word may be used often throughout the book. If you are putting this in digital format, then you want a smaller file, then you go through and put all of the words in a dictionary and give it an index number. The file will then be saved with all numbers, but later when that book is used, it will pull the correct word from the dictionary.
If we have a black and white image, we are trying to design a computer monitor in black and white. Only a single bit has to be used for each pixel as it will either be white (1) or black (0). This can be shown as: 00011000000.
Pure White is written as 1111 1111.1111 1111.1111 1111 in binary. This represents that the RGB colors are turned on to the maximum to represent white.
Blue is 0000 0000.0000 0000. 1111 1111 in binary. This shows that red is off, green is off, and blue is at its maximum.
Compression Ratio = (Original File Size in Bits - Compressed File Size in Bits) / Original File Size in Bits

Loss-Loss Techniques (must have the exact same information pull up as what was saved)

Can be used on image files.
Must be used for executables files, text, and spreadsheet data.

October 12, 2010

Practice Problem Answers:

Practice Problem 1:

1. 9 pixels * 24 bit RGB Color = 216 bits / 8 = 27 bytes
2. 4RGB 4RGB 1RGB
3 NUMBERS + 3RGB(3) = 3 bytes + 9 bytes = 12 bytes
or
3 groups * 4 bytes = 12 bytes
3. Blue = 0 Pink = 1
Color table is 0 1
Pixel Map is:
0 0 0
0 1 1
1 1 0
Answer is:
2(24) = 48 bits in color table
9 = 9 bits in pixel map
48+9 = 57 bits / 8 = 7.125 bytes

Practice Problem 2:

4. Length = 11 pixels Width = 10 pixels. Total # of pixels = 11 * 10 = 110 pixels
110 pixels * 24 bit RGB Color = 2640 bits / 8 = 330 bytes
5. 12RGB 3RGB 3RGB 3RGB 13RGB 3RGB 1RGB 1RGB 1RGB 3RGB 2RGB 3RGB 1RGB 1RGB 1RGB 3RGB 2RGB 3RGB 1RGB 1RGB 1RGB 3RGB 15RGB 1RGB 3RGB 1RGB 7RGB 3RGB 15RGB
29 NUMBER + 29 RGB(3) = 29 bytes + 87 bytes = 116 bytes
6. Green = 10, Red = 01, Yellow = 11
Color table is 10 01 11
Pixel Map is:
10 10 10 10 10 10 10 10 10 10 10
10 01 01 01 10 10 10 01 01 01 10
10 10 10 10 10 10 10 10 10 10 10
10 11 11 11 10 01 10 11 11 11 10
10 11 11 11 10 01 10 11 11 11 10
10 11 11 11 10 01 10 11 11 11 10
10 10 10 10 10 10 10 10 10 10 10
10 10 10 01 10 10 10 01 10 10 10
10 10 10 10 01 01 01 10 10 10 10
10 10 10 10 10 10 10 10 10 10 10
Answer is:
3 * 24 bits per pixel = 72 bits in color table
110 pixels in image * 2 digits per pixel in image = 220 bits in pixel map
72+220 = 292 bits / 8 = 36.5 bytes

October 14, 2010

Human Vision

Human vision system is complex. What we see with our eye is upside down until our brain processes it. Additionally, our vision is split (left and right).
Optical Illusions
http://en.wikipedia.org/wiki/Optical_illusion

JPEG Compression (Lossy)

Best suited for photographs and similar images that have fine details and continuous tones.
Think of the array of pixels as a continuous waveform with x and y with z being intensity.
High frequency components are associated with abrupt changes in image intensity.
Example: your wearing a white shirt in a picture and your friend in the same picture is wearing a black shirt. This is high changes in image intensity (the difference between black and white).
JPEG takes advantage that humans don't perceive the effect of high frequencies accurately. So basically this compression technique removes information that humans cannot see anyways.
Tries to find these high frequency components by:
Treating the image as a matrix. Each cell in matrix has a value and then attempts to remove any high frequencies in that cell.
Using Discrete Cosine Transform (DCT) to convert an array of pixels into an array of coefficients.
DCT is very computationally expensive so the image is broken into 8x8 pixel squares and applied to each of the squares.
DCT does not actually compress the image.
Encodes the frequencies at different quantization levels giving the low frequency components more quantization levels.
JPEG Compression is effective for the kinds of images it is intended for: photographic, scanned images with continuous tones, and can achieve 95% reduction in size. Allows the degree of compression. Suffers from artifacts that causes edges to blur.
Blurry edges are created from the removal of high frequency pixels that are not surrounded by lower frequency pixels.

Image Manipulation

Why?
Correct Deficiencies like red eye. Encapsulated sequence of operations to perform a particular change.
Create images that are difficult or impossible to create in nature such as special effects.
Create a WWW friendly image. Present an image in slices or in increasing resolution as it loads on the web.
Images are often organized into layers, which are like overlaid sheets that may have transparent areas. Layers are used for compositing or experimenting with different image manipulations. This prevents previous layers from being destroyed if you mess up one layer.
Composition Layers - allows you to manipulate the background and the foreground.
Most Important Tools = Selection Tools
Selection Tools allow the application of filters to only the selected parts of the image. The unaffected area is called a mask (like a stencil). A 1-bit mask is either transparent or opaque. An 8-bit mask allows 256 levels of transparency (this is know as the Alpha Channel).
For Regular Shapes: rectangular and elliptical marquee tools.
For Irregular Shapes: lasso
Magnetic Snaps (sometimes called Magic Wands) are used to select certain colors. If you use it, then it will select everything that is a similar color.

Pixel Point Processing

Allows adjustment of color in an image.
Color Adjustment - Linear
Brightness (adjusts every pixel brightness up or down) and Contrast (adjusts the range of brightness).
Color Adjustment - Nonlinear
Adjust the image histogram.
Pixel group processing is where the final value for a pixel is affected by its neighbors because the relationship between a pixel and its neighbors provides information about how color... (see slide).

Convolution and Convolution Masks

Very expensive computationally and each pixel undergoes many arithmetic operations.

Steps:

Step 1 - Verify that the convolution mask is valid (all values should add up to 1).
Step 2 - determine which pixels are involved.
Step 3 - then multiple each pixel being used by its weight in the convolution mask.
Step 4 - Add values from step three and get answer.

Example:

Convolution Mask =
1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
Image =
255 255 255 0 0
255 255 255 0 0
255 255 255 0 0
255 255 255 0 0
255 255 255 0 0

What will be the values at the location (2,2), (2,3), and (2,4)?

Location (2,2)
255 255 255
255 255 255
255 255 255

Multiplied by the convolution mask =

255(1/9) 225(1/9) 255(1/9)
255(1/9) 225(1/9) 255(1/9)
255(1/9) 225(1/9) 255(1/9)

Which becomes:

28.33 28.33 28.33
28.33 28.33 28.33
28.33 28.33 28.33

Then add up all of these values: 28.33+28.33+28.33+28.33+28.33+28.33+28.33+28.33+28.33 = 255 So location (2,2) = 255.

October 19, 2010

Chapter 5

Color and Science

Color is an essential element of multimedia that attempts to reproduce the color even though color is subjective to individual opinions.

SPD (Spectral Power Distribution) describes how the intensity of light varies with its wavelength and this alters the color that we see.

A Model of Color - Roughly Based on the Eye

The eye has:
Rods - night vision and black/white
Cones - 3 kinds: one each for red, blue, and green.
This is why we use RGB to try to reproduce this.
Tri-Stimulus Theory - the theory that states any color can be produced by mixing suitable amount of 3 additive primary colors: red, green, and blue.

RGB Color

Three primaries are standard shades of: red, green, and blue.
Any color is specified as 3 values (think Dictionary-Based Compression) giving the relative proportions of the 3 primaries. This is usually written as a 6-digit hexadecimal number with R, G, and B each being between 0 and 255.
Each hexadecimal number is comprised of 24 binary digits or bits.
In GIMP - when you click on the box on the left to select a color, you can click on any color you want. Then you can look at the HTML notation and it will provide the number for that specific color. The first 2 digits are for red, the second 2 digits are for green, and the last 2 digits are for blue.
000000 = Pure Black
ffffff = Pure White
ff0000 = Pure Red
00ff00 = Pure Green
0000ff = Pure Blue

3 Colored Things

Phosphor for a CRT and Flat panel display (aka SED or Surface Conductor Electron Emitter Display). Phosphor is made of 3 light emitting bulbs (one for red, green, and blue).
Pockets of fluorescent gas is used for Plasma.
All of them have the ability to adjust the intensity of each of the 3 things resulting in the display of most of the visible colors.

RGB

Pure Red = (100%, 0%, 0%) or (255, 0, 0)
Pure Green = (0%, 100%, 0%) or (0, 255, 0)
Pure Blue = (0%, 0%, 100%) or (0, 0, 255)


October 21, 2010

Continue Chapter 5

Mixing the Color of Light

Is an additive process which results in monitors emitting light.
Is not like mixing paint which is a subtractive process (paint absorbs light).
The number of bits used to store a color value (aka the color depth) determines how many different colors can be represented. The use of lower color depths leads to posterization and loss of image detail, but reduces file size. Too few quantization levels = posterization.
See slide (223) for an example of where the top left uses 24 bits per pixel, top right uses 8 bits per pixel, bottom left uses bits per pixel, and bottom right uses 1 bit per pixel (black and white). The fewer bits per pixel, the more blurring as it is enlarged.

Color at 16 Bit Color Depth

RGB at 24 bits
24 bits -> 3 bytes so 3 bytes with 3 colors = 1 byte per color
RGB at 16 Bits
16 bits -> 2 bytes so 2 bytes, 3 colors = 16/3 = 5 bits with one left over, with the green one getting the extra bit.
This goes back to human perception. Humans don't discriminate blue as well as they do green vision possibly due to evolutionary roots (lots of blue light in daylight and lots of green to discriminate).

Why is too big bad?

It wastes valuable computer resources (hard drive disk space, vram space, and data transit time - connection speeds on the internet).

Dithering

Some colors from the original image may be missing from the palette or color table. Dithering can be used to reduce the resulting posterization.
Dithering helps to reduce posterization to a certain extent.

End of discussion on the RGB system.

Other Color Models

CMYK Model

Cyan, Magenta, and Yellow (CMYK) are the subtractive primaries. They are the complementary colors of red, green, and blue respectively. Red is complimentary to Cyan, Green is complimentary to Magenta, and Blue is complimentary to Yellow.
Yellow = Red + Green or Yellow = White - Blue
(+) = RGB (additive) and (-) = CMYK (subtractive)
You can make black by printing all three color (CMY) but it isn't a very good black because CMY inks aren't perfect because they are chemical compounds and they are reflections. Plus it takes a long time to dry. Many printing processes are supplemented by black ink.

The HSV Model

RGB and CMYK make sense theoretically but they don't correspond to the way we experience color in the real world. You don't look at a patch of cyan color and think about the mixture of green and blue light in it; you probably relate to cyan to other blues you know (or hues), how pale it is (saturation) and its brightness (intensity or lightness).
A color can be identified by its hue, saturation, and lightness.
Tones of a single hue can be arranged two-dimensionally with lightness increasing upwards, and saturation increasing from left to right. Hues can be arranged around the rim of a color wheel with complementary colors opposite each other. Think of where you pic a color in a editing tool like GIMP (it's like a color map that blurs into all the different colors).
HSB is a variant of HSL where the tones are arranged differently.
Color spaces consisting of a brightness component and 2 color differences are used in video. They allow chrominance sub-sampling to be used.

We will finish this chapter at next class and then start on chapter 6.

October 26, 2010

Finish up Chapter 5.

Misc Info:

RGB is mainly for displaying on screens.
CMYK is mainly used for printing.
HSV (HSB & HSL ) are mainly used for hues and intensity.

Color Channels and Correction

In GIMP, you can go to "XXX" and click on "Channels", if you click the "eye" next to Red, Green, and Blue then the eye will disappear and only the colors that are selected will show up. If you select Red and Blue you will see Magenta.
Making adjustments to the channels will affect the image.
Alpha channels can be treated as an additional color channel. Images using other color models can be separated into channels, which can be processed independently.

Consistent Color

Color properties of a monitor can be roughly summarized by the RGB chromaticities, white point, and gamma.
Gamma approximately models the relationship between RGB values and light intensity.
Example: TV might ask if you are in a bright room, dark room, or moderate room and adjust the values based on this.
An image is stores in one device's color space and displayed on a device with a different color space, colors will not be reproduced accurately.
Example: If I save on a 24bit color space monitor, it will look different if I open it on a B&W screen or if I open it on a 16bit color space monitor.
If we want to make sure that an image that we save is viewed the same on another screen, then we would have to create a "source device profile". The "source device profile" tells the end user what was used when it was saved and what to view it as. This increases the file size.
ICC color profiles are one if the standard color profiles.
sRGB is intended as a standard device-independent color space for monitors and is mainly used on the WWW.
Source Device Profile is still created and is then displayed across a network or the WWW, some people may not have and Output Device Profile so then it will revert to the default color for that device and the color may still appear different.

That sums up Ch 5. Need to focus on RGB color system, how is it used, how numbers are used to describe colors (255 0 0), also know the primary application of CMYK, know the correspondence between RGB and CMYK, know what colors create what colors in both models, and know what the "K" is for on CMYK. Lastly, know that there are a couple of other models.

October 28, 2010

No notes.

Video - two different ways of making video digitally: Video or Animation.

Video - capture of frames and then playback.
Animation - create frames individually and then playback.

Video

Capture must be fast enough to capture sufficient frames to produce the illusion of motion. Transport (if across the web) and Playback must be fast enough to carry those captured frames at a rate fast enough to produce the illusion of motion.
If transport and playback are not fast enough, frames will be dropped. Video players (like Quicktime) make compromises differently:
Trying to "degrade gracefully". Some drop frames holding the last image (effectively losing the illusion of motion but continuing the story as a slide show). Some play lower resolution images. Some continue to play audio.

Video Standards: NTSC & PAL

NTSC is used in America and Japan. 29.97 frames per second. Framesize different than PAL.
PAL is used in Western Europe and Australia. 25 frames per second. Framesize different than NTSC.

November 2, 2010

Test next Thursday.

Finishing up Chapter 6 today.

Why is this a problem?

When you play them on a computer, the computer can refresh much faster and can display the entire frame at the same time.
Computer screens combine the comb effect frames and creates an overlap of the image. This seems like it would be a problem, however, it happens so quick that it hardly makes a difference. See slide 276 chapter 6 on the power point slides.
You can average the two fields and construct a single frame. This is very time consuming.
You can toss out one of the fields and interpolate between them. This gives a jumpy effect since there is no transition in between.

Digital Video Standards

Digital Standards are complicated due to:
Backward compatibility requirement - new equipment must create signals that can be handled by older equipment.
Digital from NTSC and PAL are ANALOG standards:
Each has a screen size and frame or refresh rate. The number of frames per second shown on the screen. PAL = 25, NTSC = 29.
Each define a number of lines on the screen that can be easily used for the Y dimension. PAL = 768 x 576, NTSC = 640 x 480. They have the same aspect ratio of 4:3.
Each line is a continuous (analog) signal which has to be converted to digital.
How to convert from analog to digital:
Sample the analog data - but directly sampling for each pixel results in a data stream of 20 Mbytes/second which would make it huge.
Coping with Video Size
Aside from screen size and frame rate:
Consider human vision limitations - use algebra to convert.
Human eyes are less sensitive to color changes than luminance.
Decision: Take fewer samples for color than luminance.
Without sub-sampling - for each pixel on the screen 4 things will have to be encoded: luminance, red, blue, and green.
Sub-Sampling & Understanding Human Vision
Designers realized that green contributes the most to intensity, red is next, and blue hardly contributes anything to luminance. Based on this, it was decided to use a formula for luminance:
Y = 0.2125R + 0.7154G + 0.0721B
With this we only have 3 data elements to transmit: 25% data reduction. Y = luminance, Cb = blue chrominance, Cr = red chrominance.
Chrominance Sub-Sampling
Humans can't distinguish changes in color as well as luminance changes.
Of every 4 frames: store the luminance, only store a portion of RB.
It is written as I:R:B, based on the number of times that each occurs.
IRGB IRGB IRGB IRGB so in Chrominance Sub-Sampling it would be I = 4, R = 4, B = 4 so it is written4:4:4.
IRB I IRB I so: I = 4, R = 2, B = 2 so it is 4:2:2.
If there is one without blue like: IR I IR I would be written 4:2:0. The same thing happens if there is no red like: IB IB I I would be written : 4:0:2.
NTSC & PAL Weirdness - they are each digitized through sampling to the same screen size. This means that the pixels are not square, PAL is taller than it is wide and NTSC is wider than it is tall.
MPEG-1 originally meant for Video CD (VCD). This never got very popular and it developed into a family of standards.
MPEG-2: digital broadcast TV, studio, DVD
MPEG-4: rose from the ashes, leading format for streamed video. This is used for portable devices like iPhones, iPads, etc.
Computational Irony - digital has been touted as a way to create exact copies while analog cannot (analog suffers from generational loss and digital doesn't).
But only if you use video compression that is lossless.
In production, if several people are working on the same bit of video, make sure that they all get uncompressed video to work with. Only produce the compressed version after all the work is complete.

Coping with Video Size

Compression - two versions: Spatial and Temporal.
Spatial - treating a video as a collection of frames or still images. This takes individual images and compresses them using the techniques discussed in the bitmapped section. Doesn't result in very much compression for video and doesn't take into consideration the other frames that come before or after it. Intro-frame compression.
Temporal Compression 1 - use the difference in two frames. This is a naive approach that can result in good compression and works well with small amounts of movement.
Temporal Compression 2 - when an object moves it computes its trajectory and fills in the resulting exposed background. Only the magnitude and direction need to be remembered and this results in it being very space efficient. This isn't an easy thing to do because we cannot define a specific object, we have to define pixels that are supposed to move. We can use a 16 x 16 block for this (macroblock). We then compute all possible movements of the block within a short range, compute a vector to define that movement (like dictionary based), and store the movement vectors, compress the vectors. This only works on vector graphics.
More on Temporal Compression - needs some place to start from, can be forward or backward prediction, so we need a KeyFrames (starting point):
Pick a keyframe
compute next image from that... compute next image from that... etc...
What happens when the scene completely changes? Pick a new keyframe (this requires a powerful AI - artificial intelligence).

What does video compression?

Coder/Decoder - Codec: see http://en.wikipedia.org/wiki/Video_codec
Encodes and decodes video - they can be symmetric (it takes as long to compress as decompress) or asymmetric (it takes longer to compress/decompress than it does to decompress/compress).
We have talking about making video smaller, there are a variety of techniques to do this. There is a tradeoff between compression technique (good at making the files small) and time/expense.
MPEG-4 - designed for streams that contain video, still images, animation, textures, and 3D models. Contains methods to divide scenes into arbitrarily shaped video objects.
Other Codecs: Cinepak, Intel Indeo, and Sorenson. All use vector quantization. Codec uses a collection of these "vectors" like dictionary based compression.

END Chapter 6

November 9, 2010

No notes, it was just a class day if you needed help on your project.

November 16, 2010

Went over blender project, that's all we did. We should be going over Chapter 7 next class.

November 18, 2010

We have chapter 7 and 8 to finish.

Review first Tuesday after break. Test first Thursday after break.

Chapter 7 - Animation

Two Ways to Create Moving Images: camera and animation techniques.
Animation - "bring to life" using still images to create frames that are then played one after another. Many techniques:
Flipbook - draw each frame individually.
Cel Animation - you have a fixed background and you just need to manipulate the moving part each time.
Stop Motion - used in claymation, small changes are made and a picture is taken each time.
Claymation or Modeling Clay Manipulation - use clay to build a model and then take alot of pictures.
Mixed Cel and Film - This is like the movie "Space Jam" where actual people and objects are mixed with animation.
Animation made using these techniques can be captured one frame at a time using a camera connected to a computer, instead of on film.
Animation can be created digitally as well. Individual frames can be created in a graphics program, these individual frames are saved and then are combined to play smoothly and quickly together in one stream. Using layer comps to represent the contents of a frame can streamline the animation process.
A sequence of images can be stored in consecutively numbered image files, which can be imported into video editing programs or flash. This is like what we did with the animation in GIMP.
An animated GIF contains multiple bitmapped images in a single file. The benefit of GIF is that it is supported by many different web browsers.
GIF is only suitable for short simple animation. Animated GIFs used indexed color and lossless intra-frame-compression, whose effectiveness depends on the nature of images in the animation. Animated GIFs cannot have a soundtrack or player controls.
Cel Animation
Only have to re-create the parts that change. Ex. Snow White in 1937. Uses paintings on clear plastic. The clear plastic is needed to be able to see the background and the background can be larger than the frame and "slides" past. Disney had an army of excellent painters and their original cel animations sell for a fortune.
Another example is the "Simpson" and "Family Guy"
Simpsons uses cel animation and the first 14 episodes were hand painted. Subsequent episodes used digital-ink-and-paint to mimic hand-painted cells. Took about 6-8 months per episode originally.
South Park - pilot was cut-out animation. Subsequent episodes used computer animation that mimicked cut-outs. Added some live film in later episodes and some shadowing effects. Cut-Out is when people cut out the objects and film the cut-outs to create movement. Took about 6 weeks per episode originally.
Animation Process - create drawing by either 2D model to 2D frame or 3D model to 2D frame.
2D Model to 2D Frame: hand drawn, cel, cutout. This is a really simple process. Process - Create the image. Store the image as a frame. Repeat.
3D Model to 2D Frame: physical model manipulation, stop motion claymation, 3D computer modeling. Two approaches, both with the following: produce the model, manipulate the model, define light source, define camera position and angle.
3D Model Producing 2D Images:
Using a physical 3D model - move the model, capture the frame, repeat. Very time-consuming. Wallace and Gromit, 30 frames per day, 5 years to produce.
Using a computer-based 3D Model
Ex. Toy Story, A Bug's Life, Monsters
Build the Model (takes a lot of time for the human), Move the Model (not too bad for the human), Render Frame (time consuming for the computer but not the human).
3D Model, 2D Images
Use a physical 3D model: build model, set lighting, set camera position/angle, *make frame, move model, make frame, move model*, *repeat*.
Use a computer based 3D model in Blender - build model, define light source, define camera position/angle, define object(s) movement, render frames.
Blender - open source software, powerful but complex. Need to know acronyms and shorthand and the link to them is on the bottom of fall2007 page, which will need to be built as we go. Click HERE for tutorials on blender.
Animation and Blender
Build a model to animate - 3D vector graphics modeling, defines the structure, provides a means of manipulation, allows application of surface texture.
Specify light source.
Specify camera position, direction & movement. This can be used to zoom in/out.
Manipulate the model by specifying motion tracks.
Define Objects.
Render.
Digital Cel - 2D model, works on the idea of layers (like bitmapped image layers), one layer is the background, other layers are of the different parts that will move. By moving each part a little bit you create frames of an animation.
Digital Sprite Animation - works on the idea of layers that are automated (a sprite can be thought of as an automated layer and it's motion is driven by a program called "script"). One layer is the background. Sprites are over that background layer. Sprites can have faces. Sprites can move and show different faces as they move. Sprites are basically the characters.
Sprites - the running headless man has two legs and a torso so there are three sprites total (one for each leg and one for the torso). In this example there right leg has five faces (positions).
Key Frames - came from Disney following Ford's ideas. Break production into simpler tasks. Assign tasks to less skilled labor. At Disney, Key Frames (the most important frames) are done by skilled animators and they come at important portions of the actions or at the scene changes. Then it goes to less skilled labor who then connects the action between key frames. This process is similar to interpolation like in Blender.
When you are trying to animate something in Blender:
Create an object.
Change to animation mode.
Pick a moment on the time line.
Grab the object and place it where you want it.
Press "I" on the keyboard which will bring up the "Insert Key".
Click on "LOC" which stands for "Location Change".
Pick another moment on the time line.
Grab the object and place it where you want it to go.
Press "I" on the keyboard again.
Click on "LOC" again.
2D to 2D Computer Based Key Frame Interpolation - this is natural since model is in the computer as numbers already. Forms of interpolation:
Linear - motion follows a straight line.
Quadratic - easing in and out can be used to cause motion to gradually start or stop.
Objects or layers can be made to move along motion paths. When using motion paths, it is usually necessary to orient the moving object to the path to achieve a realistic effect.
Flash Animation - flash movies, aka SWF files (Shock Wave File), are the most popular web animation format. They are usually created in Flash, but SWFs may also be exported from other programs.
Delivering Animation - Video, Flash, and GIF.
Video - must be used for animation that will be played on TV. If animation is delivered as video a soundtrack can be added, player controls can be used and high compression can be achieved using modern video codecs. Sometimes the animation and the soundtrack may not be well synchronized which leads to the sound being off. Has the best quality.
Flash - is superior for vector animations. Should be used for interactive animation requiring scripting. Flash movie format is a compressed binary format that provides an efficient way to deliver animation over a network. Most space efficient and easiest to transfer over the network.
GIF - is best for bitmapped animations.

Tuesday, we'll be starting on Chapter 8.

November 30, 2010

Blender Project is due on the before the last day of class (Next Thursday).

Test this Thursday. Study Guide is online (Course Content).

Final Exam

Return to My Digital Media Home Page

Personal tools