Title: Sensor Identification for Digital Image Forensics Data Collection and Existing Technique Implementation Faculty: Michael

l Eckmann, Assistant Professor of Computer Science Student: Adam Steinberger, class of 2013

3. Abstract/Brief Description

The overall research project involves trying to answer the question: "Given a digital image, what camera, scanner or printer was used to capture it?" There are multiple situations that can arise that require this kind of analysis. In one situation, you may be given an image and a camera and try to answer "Did the image come from that camera?, yes or no". In a more complex situation, you may have an image and a set of n cameras and try to answer "Which of the n cameras did the image come from, if any?" We will devise classification systems that must be evaluated against existing techniques on data sets that need to be collected.

Title: Sensor Identification for Digital Image Forensics Data Collection and Existing Technique Implementation Faculty: Michael Eckmann, Assistant Professor of Computer Science Student: Adam Steinberger, class of 2013

4. Faculty Statement

I gave a guest lecture in Rachel Roe-Dale's "Linear Algebra" class in Fall 2010 in which Adam was a student. The topic was on some uses of linear algebra in computer graphics. Adam was very interested and asked questions during the lecture. He asked me about my area of research and if I was open to the idea of him working with me. His eagerness sold me. We met after class and I told him about the Summer Faculty/Student Research Program and we were both interested in applying. To give him some necessary background on computer vision, I suggested he take an independent study on "Introduction to Computer Vision" with me in Spring 2011. I had taught the course once before, so Adam is going through the course notes and readings and will implement all the projects etc. We meet once per week right now and I believe this independent study will indeed give Adam the background needed to do the summer work on this project.

The working relationship I expect to have with Adam during the summer is to guide him on what needs to be done to do proper data collection and to work with him closely on understanding and implementing the existing sensor identification techniques. To achieve this, I expect that we will start by me pointing him to reading some appropriate documents on the existing techniques and continue from there.

Adam is a declared computer science major and already has coding experience in Matlab and C++, so that's a plus. Besides his eagerness and abilities, Adam currently being a sophomore is beneficial to me since he will be around two more years after the summer during which we can continue our work together to finalize the project. Him being here two more years also gives us the opportunity to possibly work together on other projects.

This summer work will benefit Adam's goals by giving him a chance to be involved with a current, important research problem. Educationally, it will provide Adam with a clear understanding of an application of computer science, which will hopefully increase his interest in his field of study.

Signed: ___________________________ Date: _02/07/2011_ Michael Eckmann

Title: Sensor Identification for Digital Image Forensics Data Collection and Existing Technique Implementation Faculty: Michael Eckmann, Assistant Professor of Computer Science Student: Adam Steinberger, class of 2013

5. Student Statement

I first met Professor Mike Eckmann at a guest lecture he gave during Professor Rachel Roe-Dales Linear Algebra class last semester. His presentation introduced our class to Computer Graphics and its applications to Linear Algebra. I approached Professor Eckmann after class, and asked if he was looking for students interested in research for Computer Vision. After confirming our combined interest in Computer Vision, we setup meetings to discuss plans for the summer. This semester, Im registered for an independent study with Professor Eckmann as an introduction to computer vision. My efforts in this class will prepare me for research over the summer.

For the ten-week summer research term, I plan to work 40 hours per week on our computer vision research. Part of this time will be spent working closely with Professor Eckmann, and the rest of this will be spent working independently.

My plans to study computer vision coincide with my personal goals in computer science. Our studies in computer vision will formalize object-oriented classes and methods for processing images and abstracting the resulting images in different ways. Image processing and pattern recognition will give me a framework in which to apply computer science as an approach to solving real-world problems. Creatively, our work in Computer Vision will help Professor Eckmann and me to further our combined goals in Computer Vision and Forensics. Our research will help us to fingerprint different cameras based on their unique image patterns. Applications of our research goals include professional and government forensics.

Signed: ___________________________ Date: _02/07/2011_ Adam Steinberger

Title: Sensor Identification for Digital Image Forensics Data Collection and Existing Technique Implementation Faculty: Michael Eckmann, Assistant Professor of Computer Science Student: Adam Steinberger, class of 2013

6. Goals, proposed activities, and format of final outcome expected

Answer: By the time the summer program ends, we wish to have completed an image database that will be used to evaluate various sensor identification techniques. The exact number and content of the images is still to be determined, but will intentionally include a wide variety of scenes taken with different camera settings (shutter speed, aperture, iso setting, etc.) from a variety of image sensors including the following: Canon EOS 50D, Kodak EasyShare Z981, Nikon D40, Olympus SP 570UZ, Panasonic Lumix DMC FZ35, and Sony Alpha DSLRA500L, all of which are either owned by Skidmore or myself. We may use other cameras that belong to one of my collaborators (he would have to ship them to us). We need to take care to capture the same scene, with the same settings with each of the cameras.

Also, we will have implemented two or three existing sensor id techniques in Matlab or C++ and will execute them on our dataset for evaluation to see how well some of the existing techniques are at sensor identification on our dataset. Techniques that I and my outside collaborators will develop will be run on the same dataset so that we can compare our techniques against existing ones, some of which will be implemented during this summer project.

The work we do will be a portion of at least one conference or journal paper.

To summarize, we expect to produce: an image database taken with the six cameras (and possibly others) for use in evaluation of sensor id techniques

implementation of existing sensor id techniques relating to image noise (e.g. "Digital Camera Identification from Sensor Pattern Noise" by J. Lukas, J. Fridrich, and M. Goljan.) results of the existing sensor id techniques on our image database and depending on how far along my collaborators and my proposed sensor id techniques are by

the summer, results of those as well a writeup of the results that we do acquire, intended to be a part of a publication

Title: Sensor Identification for Digital Image Forensics Data Collection and Existing Technique Implementation Faculty: Michael Eckmann, Assistant Professor of Computer Science Student: Adam Steinberger, class of 2013

8. Complete Description The overall research project involves trying to answer the question: "Given a digital image, what camera, scanner or printer was used to capture it?" There are multiple situations that can arise that require this kind of analysis. In one situation, you may be given an image and a camera and try to answer "Did the image come from that camera?, yes or no". In a more complex situation, you may have an image and a set of n cameras and try to answer "Which of the n cameras did the image come from, if any?" The "if any" portion of the question is more interesting than it may seem at first glance. Basically, to tackle this problem, one could create descriptors and classifiers for each of the n cameras but how to model the negative of the n cameras is an interesting question. There are techniques that have been devised that attempt to answer these questions, but improvements are needed. A clear contribution we will make will relate to how to increased rigor in evaluation experiments so that these algorithms can be assessed. This is joint work with Dr. Anderson Rocha who specializes in digital image forensics.

Another interesting question we will explore relates to how well any of the techniques we develop will react if, as a subset of our n cameras, we have multiple instances of the same make/model camera. This is difficult because not only specific features of a particular make/model camera need to be modeled, but also specific features of an instance of a camera.

We will attempt to devise detectors/classifiers that improve upon the state-of-the-art [2]. To be able to compare our detectors against existing techniques, we must be able to perform the existing techniques [1] on our data sets. So, one of the goals of the summer research is for Adam to implement a few existing techniques that model sensor noise. Another goal is to collect data under controlled situations. We have access to a variety of cameras (Canon EOS 50D, Kodak EasyShare Z981, Nikon D40, Olympus SP 570UZ, Panasonic Lumix DMC FZ35, and Sony Alpha DSLRA500L, all of which are either owned by Skidmore or myself) that are to be used to take images of similar scenes at different times in a controlled way. This data will be our ground truth (that is, we know which images came from what make/model cameras) on which we will test our algorithms. That way, we will be able to evaluate our classifiers against existing techniques. The work we do will be a portion of at least one conference or journal paper.

Sometime in April I will travel to meet my collaborator Anderson Rocha, with funds already allocated from my sabbatical and a faculty development grant. During the week long trip, I expect to meet with, besides my colleague, police experts who will provide us with some data for analysis. This will be

necessary for the larger project.

The topics I expect Adam to read and understand, either right at the beginning of our ten weeks, or possibly this semester during the independent study he is taking include guidelines on appropriate data collection techniques and background reading on existing techniques for sensor identification. We will have implemented two or three existing sensor id techniques in Matlab or C++ and will execute them on our dataset for evaluation to see how well some of the existing techniques are at sensor identification on our dataset. Techniques that I and my outside collaborators will develop will be run on the same dataset so that we can compare our techniques against existing ones, some of which will be implemented during this summer project.

Results of the existing sensor id techniques, and possibly our proposed techniques, on our image database will be determined and written up with the intention of including it in any publication that we produce.

[1] Lukas, J. Fridrich, J., and Goljan, M., "Digital Camera Identification from Sensor Noise", IEEE Transactions on Information Security and Forensics, vol. 1(2), pp. 205-214, June 2006. [2] Rocha, A., Scheirer, W., Boult, T., and Goldenstein, S., "Vision of the Unseen: Current Trends and Challenges in Digital Image and Video Forensics", ACM Computing Surveys, ISSN 0360-0300, to appear in 2011.