Enterprise AI Analysis

Case Study: Tackling Data Imbalance with Negative Sampling

Challenge: The vast majority of call center data does not contain person names, leading to a significant data imbalance. A model trained on the full dataset tended to be overly cautious, resulting in low recall and missed entities. Conversely, training only on name-containing examples led to excessive false positives, especially with common Chinese characters like “張” (Cheung), which can also mean "a measure word."

Solution: Targeted down-sampling was applied, selectively including ambiguous negative samples that could confuse the model. This ensured a balanced representation, controlling the number of negative examples to match positive ones and maintaining an equal distribution of Chinese and English negative examples to prevent bias.

Impact: This strategic approach enabled the model to learn the patterns of rare positive entities more effectively, significantly improving recall without sacrificing precision, as validated by experimental results.

Case Study: The Advantage of Cantonese-BERT-Base

Innovation: The study significantly improved upon generic Chinese BERT models by expanding its vocabulary with Cantonese-specific characters and continuing pre-training on over 10 million Cantonese sentences from online forums, where code-mixing is prevalent. This custom pre-training focused on topics like current affairs, gossip, sports, finance, and entertainment.

Impact: This specialized Cantonese-BERT-Base demonstrated superior performance across all name types, especially in handling code-mixed text. Its pre-training on authentic Cantonese data, including frequent integration of English words, equipped it with the linguistic nuances necessary for accurate inference, outperforming its Chinese-BERT-Base counterpart.

Case Study: Cantonese Linguistic Misinterpretations

Problem: The model frequently misidentifies Cantonese discourse particles and honorifics as parts of personal names. Common misclassifications include “誒” (hesitation particle), “喇” (pragmatic assertion), and “阿” (colloquial prefix) being conflated with surnames or full names. For example, "誒小姐" (Ah Miss) or "阿先生" (Ah Sir) are incorrectly tagged as names, despite "誒" and "阿" being prefixes and "先生" standing alone as a polite address.

Example: In "咁唔該呀黃生先" (First of all, thank you, Mr. Wong), "黃生先" was incorrectly tagged as a name. Only "黃生" (Mr. Wong) is the name, while "先" is a particle. This error likely arises from the surface-form overlap with the polite form "黃先生" where "先生" is an honorific.

Lesson: This highlights a need for deeper linguistic understanding beyond surface patterns, particularly in spontaneous speech where context is crucial for disambiguation.

Case Study: Confusion with Location-Derived Proper Nouns

Problem: The model occasionally mis-classifies geographical names as personal names, especially those not covered by the rule-based location masker. Culturally significant sites like ancestral halls or private roads, if absent from government datasets, can bypass pre-processing.

Example 1: "黃維則" (Wong Wai Tsak) was incorrectly tagged as a name in "黃維則堂" (Wong Wai Tsak Tong). In reality, Wong Wai Tsak Tong is an ancestral hall, and Wong Wai Tsak is not a real person.

Example 2: "張保仔" (Cheung Po Tsai) was incorrectly tagged as a name in "張保仔路" (Cheung Po Tsai Road). Cheung Po Tsai Road is a street named after a historical pirate. Even though Cheung Po Tsai is a real person, treating it as a false positive prevents degradation of transcript readability.

Lesson: Augmenting the location gazetteer with data from mapping platforms like OpenStreetMap is crucial to minimize these false positives, maintaining privacy without compromising transcript utility.