Email System with Spam Filter

Customer: A world’s leading anti-spam service provider (Canada).

Task: The Customer requested our team to enhance the product and provide an ongoing technical support.

Solution: The team designed and developed the low level (kernel) of the incoming e-mail analysis system. Based on the kernel a new email scanner was created. To improve the filtering quality, the following functions were implemented in the system:

  • Message language auto-detection (without impairing the performance),
  • Natural language parsing
  • Collecting statistics based on the parser results.

These functional modules have served as the basis for the artificial intelligence system and containers for storing and accessing the accumulated statistics. Our team has successfully resolved the task of filtering the messages in Eastern languages (Chinese, Korean, Japanese etc.). The final application, in addition to the high email filtering quality and other services, can process over 500 messages per second with full content analysis in the integrated target system.

It also uses traditional methods of spam traffic limitation, like filtering the incoming TCP/IP traffic by the incoming IP address, which helps narrowing down the channel for unknown or suspicious traffic sources.

Along with spam filtering, the system can identify messages of some specific categories, such as phishing, mail/safe-list and bounce/backscatter auto-generated.

Technologies: C++, C, Semantic analysis