研究团队

夏杰

深圳市网安信科技有限公司CEO
毕业于武汉大学，2004年入行网安、2009年获得cissp、行业经验近20年
曾任加拿大最大电信公司Telus（研科）安全实验室负责人
作为主要技术负责人完成的项目有：
1.SAP公司旗下Sybase数据库核心代码安全审计
2.加拿大四大银行中的三家银行安全审计
3.加拿大最大的药品连锁店启康药业连锁网站安全审计
精通Windows、Linux下漏洞挖掘、代码审计、二进制逆向等多种方向
曾服务于IBM、Microsoft、加拿大政府等多位大客户

周先生

暨南大学博士研究生
资深安全研究员

X先生

某985重点大学博士

合作意向

后续还将继续更新，如有合作研究意向，可评论区联系

学习笔记

2022/9/18

学习李宏毅2021/2022春机器学习课程 P85 P87 Anomaly Detection

总结:

PPT on devserver AD_DeepLearning\04-学习资料\02-讲义 Detection (v9).pptx

课程 Machine Learning Homework 8 Anomaly Detection.pdf HW讲义

Anomaly Detection 课程内容回顾：

1, clasification -> score基于Score的threshold来判断异常。训练是基于clean正常数据。

2，maximum likelyhood + Gussian distribution 基于所有数据来训练。但是需要数据分布符合Gussian才会比较准确。有各种方法来让处理过的数据符合Gussian。

3，One-class SVM

4, Isolated forest

Addtional info:

1, ROC AUC curve:

facK9s2c8@1M7s2y4Q4x3@1q4Q4x3V1k6Q4x3V1k6W2L8W2)9J5k6i4N6A6K9$3W2H3k6h3c8A6j5g2)9J5k6h3!0J5k6#2)9J5c8Y4N6A6K9$3W2Q4x3V1k6d9k6h3y4W2K9i4k6W2M7W2)9#2k6X3!0H3k6i4u0S2N6r3W2F1k6#2)9#2k6X3y4Z5j5i4u0S2j5%4c8W2M7X3W2K6N6r3W2U0

2,

b37K9s2c8@1M7s2y4Q4x3@1q4Q4x3V1k6Q4x3V1k6K6M7r3g2W2j5$3S2Q4x3X3g2W2k6g2)9J5k6h3&6@1N6g2)9J5k6h3g2V1N6g2)9J5k6i4c8%4i4K6u0r3i4K6N6q4K9s2W2D9k6h3g2Q4x3V1k6E0L8q4)9J5c8X3#2D9x3U0l9J5x3W2)9J5k6r3y4G2N6i4u0K6k6g2)9J5k6r3c8S2N6r3q4Q4x3V1k6y4j5h3y4Z5K9h3&6W2i4K6t1#2x3U0m8x3k6h3q4J5L8X3W2F1k6#2)9J5y4e0t1H3d9r3!0E0k6i4N6G2M7X3E0Q4x3U0f1J5x3o6S2Q4x3U0f1J5x3p5q4F1L8$3#2S2L8s2W2Q4x3U0f1J5x3p5c8W2N6r3g2U0N6r3W2G2L8W2)9J5k6i4m8V1k6R3`.`.

因为我们的数据集是大量的polluted数据，label数据比较少。所有 clasification -> score 这种方式不太时候。监督学习方式都不太适合。

下一步看无/自监督学习。

HW8: code:

689K9s2c8@1M7s2y4Q4x3@1q4Q4x3V1k6Q4x3V1k6U0L8$3I4S2j5W2)9J5k6i4u0W2M7$3g2S2M7X3y4Z5i4K6u0W2k6$3!0G2k6$3I4W2i4K6u0W2j5$3!0E0i4K6u0r3k6$3W2@1K9s2g2T1i4K6u0r3k6$3p5$3y4o6t1K6z5o6q4Q4x3V1k6y4e0o6t1H3x3U0q4Q4x3X3c8e0M7s2u0A6L8X3N6Q4x3V1k6T1L8r3!0T1i4K6u0r3L8h3q4A6L8W2)9J5c8V1S2i4x3o6S2Q4x3V1k6t1g2K6l9^5i4K6u0W2K9i4m8&6L8X3u0Q4x3U0y4K6j5%4u0G2L8r3I4f1L8#2)9K6c8p5c8o6k6@1&6j5f1%4y4q4g2%4g2k6y4H3`.`.

Dataset: login 之后，点击左侧文件， 双击 Set up the environment，会出现对应数据集相关部分。具体下载地址如下：

64cK9s2c8@1M7s2y4Q4x3@1q4Q4x3V1k6Q4x3V1k6V1M7X3W2$3k6g2)9J5k6h3N6G2L8$3N6D9k6g2)9J5k6h3y4G2L8g2)9J5c8X3k6A6L8r3g2Q4x3V1k6V1i4K6u0r3x3e0g2j5g2@1!0Q4x3X3c8*7d9g2)9J5k6p5q4w2g2K6m8A6k6$3k6%4f1%4W2V1L8i4N6e0c8$3p5^5c8f1&6T1z5i4N6o6k6#2)9J5c8X3g2V1K9i4b7`.

已下载到：/home/wax/AD_DeepLearning/03-数据集/01-HW8

TODO:

1, Study auto-decoder.

2, HW8 change to use GPU, study HW8 code. - done

2022/9/18

1, Run HW8 on different network. If need to change cnn to fcn, need to change model_type and checkpoint_path.

2, download npyviewer and change the code to be able to write dataset to jpeg. The other format is wrong. study auto-decoder 1.1 Feature disentangle

TODO:

1, learn 机器学习的可解释性

2, Learn 自监督学习

3，Anormaly detection zhihu:

8c4K9s2c8@1M7s2y4Q4x3@1q4Q4x3V1k6Q4x3V1k6*7K9s2g2S2L8X3I4S2L8W2)9J5k6i4A6Z5K9h3S2#2i4K6u0W2j5$3!0E0i4K6u0r3M7q4)9J5c8U0b7J5x3U0j5%4y4U0f1J5

4, Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark - a benchmark for anomaly detection.

9dcK9s2c8@1M7s2y4Q4x3@1q4Q4x3V1k6Q4x3V1k6Y4K9i4c8Z5N6h3u0Q4x3X3g2U0L8$3#2Q4x3V1k6F1N6h3#2W2L8Y4c8S2i4K6u0r3e0V1q4n7

5,

076K9s2c8@1M7s2y4Q4x3@1q4Q4x3V1k6Q4x3V1k6%4N6%4N6Q4x3X3g2C8j5h3N6Y4L8r3g2Q4x3X3g2U0L8$3#2Q4x3V1k6V1j5i4c8S2M7$3g2@1M7#2)9J5c8X3#2D9k6#2)9J5k6s2g2D9j5W2)9J5c8X3y4J5k6h3c8A6N6r3y4S2M7X3c8X3M7X3q4#2k6l9`.`.

-- creditcard fraud detection. dataset already downloaded. Book reference

bdfK9s2c8@1M7s2y4Q4x3@1q4Q4x3V1k6Q4x3V1k6X3M7X3q4#2k6q4)9J5k6r3c8W2N6r3g2U0N6r3W2G2L8W2)9J5k6r3S2S2L8X3c8T1L8$3!0C8i4K6u0W2k6$3W2@1K9s2g2T1i4K6u0W2K9h3!0Q4x3V1k6X3M7X3q4#2k6q4)9J5k6r3c8W2N6r3g2U0N6r3W2G2L8W2)9J5k6r3S2S2L8X3c8T1L8$3!0C8i4K6u0r3c8X3!0J5k6i4N6G2M7X3c8Q4x3X3g2Z5N6r3#2D9

[培训]科锐逆向工程师培训第53期2025年7月8日开班！