The Searching CIT Test

What a Searching CIT Is

Sometimes called the searching concealed information test, or SCIT. The idea is simple; the difficulties are in the detail.

In the standard Concealed Information Test, described on the how CIT testing works page, the person arranging the test already knows a true detail of the event. The test checks whether the tested person recognises it. The examiner knows which item on the list is real, and the analysis asks one question: is the response to that item larger than the response to the others?

There are cases where that is not possible because the critical fact is unknown. Something was taken, but nobody knows where it is now. Something happened, but nobody knows where, or with whom, or on which day. The only person who might know is the person to be tested.

The searching CIT turns the method around for these cases. Instead of one known probe among invented alternatives, the list is made of candidate answers: every reasonable possibility for the unknown fact. The tested person sees them one at a time, exactly as in the standard test, while EEG is recorded. The analysis then asks a different question: does any one of these candidates draw a response that stands apart from the rest?

If one does, it becomes a lead: a suggestion of which candidate is meaningful to the person. If none does, the outcome is that nothing on the list stood out.

Is This a Recognised Method?

Yes. It was not invented for this page, and its limits are documented alongside its uses.

The distinction between the two forms of the test is well established in the research literature. A 2012 review in Frontiers in Psychology by Izumi Matsuda, Hiroshi Nittono and John Allen, describing the use of the CIT by the Japanese police, sets them out side by side. In the known-solution CIT, the investigators already know the critical fact and want to learn whether the examinee does. In the searching CIT, the investigators do not know the fact and use the examinee's responses to look for it, for example to learn where something has been hidden.

The same review reports that Japanese police examiners, of whom there are around 100 carrying out about 5,000 CIT examinations a year, use both forms in practice. Their examinations rely on bodily measures such as skin conductance, heart rate and breathing, not on EEG. The review is also clear that the searching form is the harder of the two to interpret. With no known correct answer, the examiner cannot check the result against the truth, and a result in which no item stands out cannot be taken as evidence that the examinee knows nothing. It may simply mean that the correct answer was not among the items.

P300 versions of the searching approach have been studied in laboratories as well, typically by having participants conceal one item from a set and testing whether the analysis can pick it out without being told which it was. It works above chance, and it works less well than the known-answer test, for reasons the next section explains.

Why Searching Is Harder Than Checking

The reason is statistical, and it matters enough to explain properly.

Take six items and record a person's brain response to each, thirty times over. Average the responses per item. Even if the person has no knowledge at all, the six averages will not be identical. Background brain activity is random, so one of the six will come out largest purely by chance. There is always a winner.

In the standard CIT this is not a problem, because the item being examined was fixed in advance. The question "is the probe larger than the others?" was asked before the data came in. If the probe is no different from the rest, it has only a one-in-six chance of being the largest, and a much smaller chance of being consistently and clearly the largest.

In a searching test, nobody fixed an item in advance. The analysis looks at all six, takes whichever is largest, and asks whether it is special. But it was selected because it was largest. Compare the winner with the others in the ordinary way and it will look impressive nearly every time. A method that did this without correction would find a "recognised" item in almost every test of every person.

There is a second difficulty. In the standard test, five irrelevants are pooled against one probe, giving a steady baseline. In a searching test every item is a suspect, so each has to be compared with the others while any of them might be the real one. There is less data behind each comparison.

And a third. In many searching cases, the candidates are not neutral. If the list is the names of six people the tested person knows, all six are familiar. The test is then looking for one familiar thing that is more significant than five other familiar things, which is a smaller difference to detect than known against unknown.

How We Score a Searching Test

The analysis is built to answer one question honestly: is the winner further ahead than a chance winner would be?

1. The recording is prepared as in the standard test. Filtering, cutting into epochs, rejecting epochs contaminated by movement, and measuring the P300 at Pz for each item.
2. The target is checked. Each block still contains a target item with a key-press task. If the target does not produce a clear P300, the block is unusable and is set aside.
3. The lead is measured. We take the item with the largest P300 and calculate how far it is ahead of the average of the other items.
4. The labels are shuffled. The software takes all the recorded responses to the candidate items and randomly reassigns which item each one belongs to. That destroys any real difference between items while keeping the noise exactly as it was. It then finds the largest item in the shuffled data and measures its lead in the same way.
5. The shuffle is repeated 1,000 times. This produces a picture of how big a lead the winner gets when there is truly nothing to find.
6. The real lead is compared with the shuffled ones. If the real lead is bigger than all but 5 per cent of the shuffled leads, the item is recorded as standing out in that block.

This is called a permutation test. Its important property is that the comparison is like for like. The real data and the shuffled data are both scored by taking the maximum, so the advantage of picking the winner afterwards is cancelled out.

One block is not enough

Even with that correction, a threshold of 5 per cent means that about one block in twenty will show a stand-out by chance alone. So we do not report a pattern on the strength of a single block. A searching test is built from several blocks that approach the unknown fact from different directions, and the overall outcome is one of the following:

We also use more repetitions than in the standard test, thirty showings of each item as a default, because our own bench work showed that twenty was too few for a stand-out to clear the threshold reliably.

Building the Candidate Lists

The list defines what the test can possibly find.

A searching block needs a question with a limited set of answers. "Where is it?" works if the possibilities can be narrowed to five or six places. "Who was involved?" works if there are five or six plausible people. "What happened?" does not work, because it has no list.

The rules for a sound list are stricter than they first appear:

• The true answer must be on it. If it is not, the test cannot find it, and the result will be "nothing stood out" however much the person knows.
• The candidates must be comparable. Same type, similar length, similar familiarity. A list of five colleagues and one stranger will make the stranger look different for the wrong reason.
• No candidate should carry extra weight for an unrelated reason. A person's best friend, their home town or their own street will draw a response regardless of the case.
• The list should not be too long. Each item needs around thirty showings. Six candidates plus a target is a block of about eight minutes. Ten candidates would be too long to sit through with a clean recording.

Because one block is not enough, we look for two or three different questions that would all point the same way if the suspicion were right. For example, one block on a person, one on a place and one on a day. A pattern is reported only when stand-outs appear in at least two of them.

When a Searching Test Is Considered

It is a second choice. We suggest it only where the standard test cannot be built.

Locating something

This is the classic field use described in the Japanese literature: an item is missing and the question is where it went. If the possibilities can be listed, and the tested person would know the answer if involved, a searching block can be built around locations.

Narrowing down an unknown

In a workplace matter there may be no doubt that something was disclosed, but uncertainty about how: by email, on paper, by photograph, by word of mouth. A list of methods is a workable set of candidates.

Relationship cases without a known detail

Some people who suspect a partner have no specific name or place to test. A searching test can present candidate names or places. This is the hardest use, because every candidate drawn from someone's own life is already familiar to them. We discuss it frankly on the page about CIT testing for a suspected cheating partner, and in many such cases we advise against testing.

As a demonstration

The searching format is also how a "pick a card" style demonstration works. A person privately chooses one item from a set and the test tries to identify it. That is useful for showing the method working and for our own validation, and it is the situation in which the searching test performs best, because the chosen item was neutral beforehand and is freshly significant.

A Worked Example

An invented workplace case, to show how the pieces fit.

A small firm discovers that a sample product has gone from a locked store. Four people had access. The owner does not know where the sample is now or how it left the building, so there is no known detail to build a standard test on. One member of staff has agreed to be tested.

Three searching blocks are drafted. The first lists six ways the sample could have left: in a rucksack, in a delivery box, in a coat, in a car boot, in a waste sack, in a toolbox. The second lists six places it might be now. The third lists six days of the fortnight in question. Each block has a target item for the key-press task.

Suppose the recording shows "car boot" standing out in the first block at the 5 per cent threshold, a particular location standing out in the second, and nothing in the third. Two usable blocks have a stand-out, so the outcome is reported as a consistent pattern, with the two items named as leads. If only "car boot" had stood out, the report would say that one item stood out and that this is not enough on its own. In neither case would the report say what the person did.

What a Searching Result Does Not Tell You

Read this section before deciding whether the test is worth having.

• "Nothing stood out" is not a clearance. It means no candidate on these lists drew a response beyond chance in this recording. The true answer may not have been listed. The person may know nothing. The difference may have been too small to detect. The test cannot tell these apart.
• A stand-out is a lead, not a finding of fact. It indicates that an item is more significant to the person than the alternatives. It does not say why. A name may stand out because of the matter in question, or because of a recent argument, a bereavement, or an association nobody thought of.
• There is no known answer to check against. In the standard test, the probe is fixed, so the logic is closed. In a searching test, if an unexpected item stands out, nobody can say from the test alone whether that is a discovery or an accident.
• Inconclusive outcomes are common. In our own bench recordings with a known concealed item, the correct item came out on top each time, but it cleared the statistical threshold in only a minority of blocks at lower repetition counts. That is why we raised the default repetitions and require two blocks.
• The same vulnerabilities apply. A person can be taught techniques that weaken a P300 test; research in 2004 demonstrated this for the three-stimulus protocol we use. And like the standard CIT, the searching test has limited field validation with EEG.

The published laboratory figures for P300 CIT accuracy, commonly 80 to 95 per cent detection of people holding the tested information, come from known-answer tests. They are not results from our own testing, and they should not be assumed to carry over to the searching form, which the literature treats as the more difficult case.

What the Appointment Involves

For the tested person, a searching test feels the same as a standard one, only a little longer.

The examiner takes written consent, fits the four-channel EEG cap and checks the signal. Each block begins with an instruction naming a target item and the keys to press. Words then appear one at a time, each for about a third of a second, at roughly one every two seconds. The person presses Y for the target and N for everything else.

Before each block there is a short pause while the examiner introduces the next list, and there are rests within blocks. With two or three blocks at thirty repetitions, the recording itself takes 25 to 35 minutes, and the appointment as a whole around 90 minutes.

The person is not asked any accusing questions and is not told what the lists are intended to find. At the end, the screen shows a neutral completion message. The examiner reviews the recording quality, and the outcome is reported block by block with the figures behind it. The complete raw recording is retained so the analysis can be repeated.

Deciding Between the Two Forms

A short guide.

If you are unsure which applies, the who qualifies page has a checklist, and the CIT testing overview covers the standard test. When you book, tell us what is known and what is not, and we will say which form, if either, is worth running.