Pattern frequency statistics

These two indicators give patterns by converting alpha characters to a and numerics to 9.

Pattern frequency indicators and database compatibility

The following table shows the indicators that you can select in any database:

These two indicators work only with Latin characters and are available only with the Java engine. They are useful when you want to identify patterns in Asian data.

The above two indicators give patterns by converting Asian characters to letters such as H,K,C and G following the rules described in the following table:

Character type	Usage
Latin numbers	9 replaces all ASCII digits
Latin lowercase letters	a replaces all ASCII Latin characters
Latin uppercase letters	A replaces all uppercase Latin characters
Full-width Latin numbers	9 replaces all ASCII digits
Full-width Latin lowercase letters	a replaces all ASCII Latin characters
Full-width Latin uppercase letters	A replaces all uppercase Latin characters
Hiragana	H replaces all Hiragana characters
Half-width Katakana	k replaces all half-width Katakana characters
Full-width Katakana	K replaces all full-width Katakana characters
Katakana	K replaces all Katakana characters
Kanji	C replaces Chinese characters
Hangul	G replaces Hangul characters

Below is an example of a column analysis using the East Asia Pattern Frequency and East Asia Pattern Low Frequency indicators on an address column.

The analysis results of the East Asia Pattern Low Frequency indicator will look like the following:

These results give the number of the least frequent records for each distinct pattern. Some patterns have characters and numbers, while others have only characters. Patterns also have different lengths, so this shows that the address is not consistent and you may need to correct and clean it.

East Asia pattern frequency indicators and database compatibility

The following table shows the indicators that you can select in any database:

Date pattern frequency indicator and database compatibility

The following table shows the indicators that you can select in any database:

Case-sensitive indicators

Patterns focus on words and are case sensitive:

Pattern	Description
[Word]	Word starting with an uppercase character and consisting of lowercase characters
[WORD]	Word with uppercase characters
[word]	Word with lowercase characters
[Char]	Single uppercase character
[char]	Single lowercase character
[Ideogram]	One of the CJK Unified Ideographs
[IdeogramSeq]	Sequence of ideograms
[hiraSeq]	Sequence of Japanese Hiragana characters
[kataSeq]	Sequence of Japanese Katakana characters
[hangulSeq]	Sequence of Korean Hangul characters
[digit]	One of the Arabic numerals: 0,1,2,3,4,5,6,7,8,9
[number]	Sequence of digits

When using the CS Word Pattern Frequency and CS Word Pattern Low Frequency indicators, the following strings are replaced with the following patterns:

String	Pattern
A character is NOT a Word	[Char] [word] [word] [WORD] [char] [Word]
someWordsINwORDS	[word][Word][WORD][char][WORD]
Example123@domain.com	[Word][number]@[word].[word]
anotherExample8@domain.com	[word][Word][digit]@[word].[word]
袁 花木蘭88	[Ideogram] [IdeogramSeq][number]
Latin2中文	[Word][digit][IdeogramSeq]
Latin3フランス	[Word][digit][kataSeq]
Latin4とうきょう	[Word][digit][hiraSeq]
Latin5나는 한국 사람입니다	[Word][digit][hangulSeq]

Case-insensitive indicators

Patterns focus on words and are case insensitive:

Pattern	Description
[word]	Word with lowercase characters
[char]	Single lowercase character
[Ideogram]	One of the CJK Unified Ideographs
[IdeogramSeq]	Sequence of ideograms
[hiraSeq]	Sequence of Japanese Hiragana characters
[kataSeq]	Sequence of Japanese Katakana characters
[hangulSeq]	Sequence of Korean Hangul characters
[digit]	One of the Arabic numerals: 0,1,2,3,4,5,6,7,8,9
[number]	Sequence of digits
[alnum]	Alphanumeric value consisting of characters and Arabic numerals

When using the CI Word Pattern Frequency and CI Word Pattern Low Frequency indicators, the following strings are replaced with the following patterns:

String	Pattern
A character is NOT a Word	[char] [word] [word] [word] [char] [word]
someWordsINwORDS	[word]
Example123@domain.com	[alnum]@[word].[word]
anotherExample8@domain.com	[alnum]@[word].[word]
袁 花木蘭88	[Ideogram] [IdeogramSeq][number]
Latin2中文	[word][digit][IdeogramSeq]
Latin3フランス	[word][digit][kataSeq]
Latin4とうきょう	[word][digit][hiraSeq]
Latin5나는 한국 사람입니다	[word][digit][hangulSeq]

Word-based pattern indicators and database compatibility

The following table shows the indicators that you can select in any database: