Semi-supervised learning via manifold regularization

doi:10.1016/S1005-8885(11)60321-X

Acta Metallurgica Sinica(English letters) ›› 2012, Vol. 19 ›› Issue (6): 79-88.doi: 10.1016/S1005-8885(11)60321-X

Semi-supervised learning via manifold regularization

毛昱,周延泉

北京邮电大学

收稿日期:2012-05-07 修回日期:2012-07-02 出版日期:2012-12-31 发布日期:2012-12-14
通讯作者: 毛昱 E-mail:maomaoyu10@gmail.com
基金资助:
This work was supported by the Mechanism Socialist Method and Higher Intelligence Theory of the National Natural Science Fund Projects (60873001).

Semi-supervised learning via manifold regularization

Received:2012-05-07 Revised:2012-07-02 Online:2012-12-31 Published:2012-12-14
Supported by:
This work was supported by the Mechanism Socialist Method and Higher Intelligence Theory of the National Natural Science Fund Projects (60873001).

摘要/Abstract

摘要：

This paper proposes a novel graph-based transductive learning algorithm based on manifold regularization. First, the manifold regularization was introduced to probabilistic discriminant model for semi-supervised classification task. And then a variation of the expectation maximization (EM) algorithm was derived to solve the optimization problem, which leads to an iterative algorithm. Although our method is developed in probabilistic framework, there is no need to make assumption about the specific form of data distribution. Besides, the crucial updating formula has closed form. This method was evaluated for text categorization on two standard datasets, 20 news group and Reuters-21578. Experiments show that our approach outperforms the state-of-the-art graph-based transductive learning methods.

关键词:

manifold regularization, semi-supervised learning, transductive learning, expectation maximization algorithm, classification, text categorization

Abstract:

Key words:

manifold regularization, semi-supervised learning, transductive learning, expectation maximization algorithm, classification, text categorization

参考文献

1. Corduneanu A, Jaakkola T. On information regularization. Proceedings of the 19th Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI’03), Aug 7-10, 2003, Acapulco, Mexico. San Francisco, CA, USA: Morgan Kaufmann, 2003: 151-158

2. Gideon S M, McCallum A. Simple, robust, scalable semi-supervised learning via expectation regularization. Proceedings of the 24th Annual International Conference on Machine Learning (ICML’07), Jun 20-24, 2007, Nashville, Corvallis, OR, USA. New York, NY, USA: ACM, 2007: 593-600

3. Grandvalet Y, Bengio Y. Semi-supervised learning by entropy minimization. Advances in Neural Information Processing Systems: Proceedings of the 18th Annual Conference on Neural Information Processing Systems (NIPS’04), Dec, 13-18, 2004, Vancouver, Canada. Cambridge, MA, USA:Cambridge University Press, 2004: 529-536

4. Belkin M, Niyogi P, Sindhwani M. Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. The Journal of Machine Learning Research, 2006, 7: 2399-2434

5. Belkin M, Niyogi P. Laplacian eigenmaps and spectral techniques for embedding and clustering. Advances in Neural Information Processing Systems: Proceedings of the 15th Annual Conference on Neural Information Processing Systems (NIPS’01), Dec, 3-8, 2001, Vancouver, Canada. Cambridge, MA, USA:Cambridge University Press, 2001: 585-591

6. Blum A, Chawla S. Learning from labeled and unlabeled data using graph mincuts. Proceedings of the 18th Annual International Conference on Machine Learning (ICML’01), Jun 28-Jul 1, 2001, Williamstown, MA, USA. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc, 2001: 19-26

7. Cai D, Wang X H, He X F. Probabilistic dyadic data analysis with local and global consistency. Proceedings of the 26th Annual International Conference on Machine Learning (ICML’09), Jun 14-18, 2009, Montreal, Canada. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc, 2009: 105-112

8. Joachims T. Transductive learning via spectral graph partitioning. Proceedings of the 20th Annual International Conference on Machine Learning (ICML’03), Aug 21-24, -2003, Washington, DC, USA. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc, 2003: 290-297

9. Subramanya A , Bilmes J. Soft-supervised text classification. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’08), October 25-27, 2008, Honolulu, HI, USA. New York, NY, USA: ACM, 2008: 1090-1099

10. Subramanya A, Bilmes J. Entropic graph regularization in non-parametric semi-supervised classification. Advances in Neural Information Processing Systems: Proceedings of the 23rd Annual Conference on Neural Information Processing Systems (NIPS'09), Dec 9-14, 2009, Vancouver, Canada. Cambridge, MA, USA:Cambridge University Press, 2009 : 1803-1811

11. Mao Y, Xi M Y, Yu H, et al. Semi-supervised logistic regression via manifold regularization. In Proceedings of the 2011 IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS’11), Sep 15-17, 2011, Beijing, China. Piscataway, NJ, USA: IEEE, 2011: 23-28

12. Zhou D, Bousquet O, Lal T, et al. Learning with local and global consistency. Advances in Neural Information Processing Systems: Proceedings of the 18th Annual Conference on Neural Information Processing Systems (NIPS’04), Dec, 13-18, 2004, Vancouver, Canada. Cambridge, MA, USA:Cambridge University Press, 2004: 321-328

13. Zhu X, Lafferty J. Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning. Proceedings of the 22nd Annual International Conference on Machine Learning (ICML’05), Aug 7-11, 2005, Bonn, Germany. New York, NY, USA: ACM, 2005: 1052-1059

14. Zhu X. Semi-supervised learning with graphs. Ph. D. Thesis. Madison, WI, USA: University of Wisconsin-Madison, 2005.

15. Ulrike L. A tutorial on spectral clustering. Statistics and Computing, 2007, 17(4): 395-416

16. Zhu X. Semi-supervised learning literature survey. TR-1530. Madison, WI, USA: Computer Sciences, University of Wisconsin-Madison, 2005

17. Dempster A P, Laird N M, Rubin D B. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B, 1977, 39(1): 1-38

18. Chang C C, Lin C J. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2011, 2(3): Article 27

Semi-supervised learning via manifold regularization

Semi-supervised learning via manifold regularization

可视化

被引次数

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价