Engineer friends often ask me: Graph Deep Learning sounds great, but are there any big commercial success stories? Is it being deployed in practical applications? Besides the obvious ones–recommendation systems at Pinterest, Alibaba and Twitter–a slightly nuanced success story is the Transformer architecture, which has taken the NLP industry by storm. Through this post, I want to establish links between Graph Neural Networks (GNNs) and Transformers. I’ll talk about the intuitions behind model architectures in the NLP and GNN communities, make connections using equations and figures, and discuss how we could work together to drive progress.
A. Asif. International Journal of Innovative Research in Information Security, 8 (3):
27-33(August 2021)1. Abdulkadir Sengur, “An expert system based on linear Discriminant analysis and adaptive neuro-fuzzy inference system to diagnosis heart valve diseases”, ELSEVIER, Expert Systems with Applications 35, 214-222, 12 June 2007. 2. Jain Yu, “General C-Means Clustering Model”, IEEE Transactions On Pattern Analysis And Machine Intelugence, VOL. 27, NO. 8, August 2005. 3. Meyer-Baese, O. Lange, A. wismueller, and M. K. Hurdal, “Analysis of Dynamic Susceptibility constrast MRI Time Series Based on Unsupervised Clustering Methods”, IEEE Transactions on Information Technology in Biomedicine, VOL. 11, NO. 5, September 2007. 4. Shaikh Abdul Hannan, "Heart Disease Diagnosis by using FFBP and GRNN algorithm of Neural Network", International Journal of Computer Science and Information Security, Vol 12, Number 6, June 2014, ISSN 1945-5500, United States of America. 5. Fraser, H.S.F., et al., “Differential diagnoses of the heart disease program have better sensitivity than Resident Physicians”, Tufts-New England Medical Center, Boston, MA(2001). 6. Mohammad Eid Alzaharani, Shaikh Abdul Hannan, “Diagnosis and Medical Prescription of Heart Disease Using FFBP, SVM and RBF”, Issue,1, Vol 5, , KKU Journal of Basic and Applied Sciences, Mar 2019 , Page 6-15. 7. Frawley and Piatetsky-Shapiro, 1996. Knowledge Discovery in Databases: An Overview. The AAAI/MIT Press, Menlo Park, C.A. 8. Aqueel Ahmed, Shaikh Abdul Hannan, “Data Mining Techniques to Find Out Heart Diseases: An Overview”, International Journal of Innovative Technology and Exploring Engineering (IJITEE), An ISO 9001:2008 Certified International Journal, Volume-1, Issue-4, September 2012, ISSN: 2278-3075, New Delhi, India. 9. I. Turkoglu, A. Arslan, E. Ilkay, “An expert system for diagnosis of the heart valve diseases”, Expert Systems with Applications vol.23, pp. 229–236, 2002. 10. I. Turkoglu, A. Arslan, E. Ilkay, “An intelligent system for diagnosis of heart valve diseases with wavelet packet neural networks”, Computer in Biology and Medicine vol.33 pp.319–331, 2003. 11. Haque ME, Sudhakar KV. ANN back propagation prediction model for fracture toughness in microalloy steel. Int J Fatique 2002;24:1003–10. 12. Shaikh Abdul Hannan, A.V. Mane, R. R. Manza and R. J. Ramteke, “Prediction of Heart Disease Medical Prescription Using Radial Basis Function", IEEE International Conference on Computational Intelligence and Computing Research at Tamilnadu College of Engineering, Coimbatore, Tamilnadu, India, ICCIC-2010, December 28-29, 2010. 13. P.I.J. Keeton, F.S. Schlindwein, Application of Wavelets in Doppler Ultrasound, vol. 17, number 1, MCB University Press, 1997, pp. 38–45. 14. I.A. Wright, N.A.J. Gough, F. Rakebrandt, M. Wahab, J.P. Woodcock, Neural network analysis of Doppler ultrasound blood low signals: a pilot study, Ultrasound in Medicine & Biology 23 (5) (1997) 683–690. 15. I. Guler, M.K. Kiymik, S. Kara, M.E. Yuksel, Application of autoregressive analysis to 20 MHz pulsed Doppler data in real time, International Journal Biomedical Computing 31 (3–4) (1992) 247–256. 16. Shaikh Abdul Hannan, "Heart Disease Diagnosis by using FFBP and GRNN algorithm of Neural Network", International Journal of Computer Science and Information Security, Vol 12, Number 6, June 2014, ISSN 1945-5500, United States of America. 17. Santosh K. Maher, Sumegh Tharewal, Abdul Hannan, “Review on HRV based Prediction and Detection of Heart Disease”, International Journal of Computer Applications (0975 – 8887), Pag 7-12, Volume 179 – No.46, June 2018. 18. https://www.who.int/en/news-room/fact sheets/detail/cardiovascular-diseases-(cvds) https://www.mayoclinic.org/diseases-conditions/stroke/symptoms-causes/syc-20350113. 19. Shaikh Abdul Hannan, “An Overview of Big Data and Hadoop”, International Journal of Computer Application”, Volume 154, Number 10, ISSN – 0975-887, November 2016, New York, USA. 20. Santosh Maher, Shaikh Abdul Hannan, Sumegh Tharewal, K. V. Kale " HRV based Human Heart Disease Prediction and Classification using Machine Learning " December 2019, (Vol. 17 No. 2 International Journal of Computer Science and Information SecApplicatiion (IJCA), New York, USA. 21. Akram Ablsubari, Shaikh Abdul Hannan, Mohammed Eid Alzaharani, Rakesh Ramteke, "Composite Feature Extraction and Classification for Fusion of Palmprint and Iris Biometric Traits", Engineering Technology and Applied Science Research, (ETASR) Volume 9, No 1, Feb 2019, ISSN: 2241-4487, Greece. 22. Yogesh Rajput, Shaikh Abdul Hannan, Mohammed Eid Alzaharani, D. Patil Ramesh Manza, Design and Development of New Algorithm for person identification Based on Iris statistical features and Retinal blood Vessels Bifurcation points” ” International Conference on Recent Trends in Image Processing & Pattern Recognition (RTIP2R), December 21-22, 2018, India. 23. Yogesh Rajput, Shaikh Abdul Hannan, “Design New Wavelet Filter for Detection and Grading of Non-proliferative Diabetic Retinopathy Lesions”, International Conference on Recent Trends in Image Processing and Pattern Recognition, Jan 2020, Springer, Singpore. 24. Y. M. Rajput, A. H. Hannan, M. E. Alzahrani, R. R. Manza, D. D. Patil, “EEG-Based Emotion Recognition Using Different Neural Network and Pattern Recognition Techniques–A Review”, International Journal of Computer Sciences and Engineering, Vol 6, Issue 9, Sep 2018. 25. Shaikh Abdul Hannan, Bharatratna P. Gaikwad, Ramesh Manza, "Brain Tumor from MRI Images : A Review". International Journal of Scientific and Engineering Research (IJSER), Volume 5, Issue 4, April-2014 ISSN 2229-5518, France. 26. Eliane Rich and KevinKnight – Artificial Intelligence – Secone Edition Mcgraw Hill , 1983. 27. Long, W., et.al., “Developing a program for Tracking Heart Failure”, MIT Lab for Computer Science, Cambridge, MA(2001). 28. Frase, H.S.F., et.al., “Comparing complex diagnoses a formative evalution of the heart disease program”, MIT Lab for Computer Science, Cambridge, MA(2001) 29. Shaikh Abdul Hannan, A.V. Mane, R. R. Manza and R. J. Ramteke, “Prediction of Heart Disease Medical Prescription Using Radial Basis Function", IEEE International Conference on Computational Intelligence and Computing Research at Tamilnadu College of Engineering, Coimbatore, Tamilnadu, India, ICCIC-2010, December 28-29, 2010. 30. J. Galindo, P. Tamayo, Credit risk assessment using statistical and machine learning: basic methodology and risk modeling applications, Computational Economics 15 (1 – 2) (2000) 107–143 31. V. Vapnik, The Nature of Statistical Learning Theory, Springer- Verlag, New York, 1995. 32. M.A. Hearst, S.T. Dumais, E. Osman, J. Platt, B. Scho¨lkopf, Support vector machines, IEEE Intelligent Systems 13 (4) (1998) 18– 28. 33. Broomhead D., & Lowe, D., Multivariable functional interpolation and adaptive networks. Complex Systems, vol.2, pp.321-355, 1988. 34. Haralambos Sarimveis, Philip Doganis, Alex Alexandridis, “A classification technique based on radial basis function neural networks”, Advances in Engineering Software vol.37, pp.218–221, 2006. 35. Shaikh Abdul Hannan, Pravin Yannawar, R.R. Manza and R.J. Ramteke, “Expert System Data Collection Technique for Heart Disease” , in IT & Business Intelligence, on 06-08 Nov 2009, Organised By IMT, Nagpur, India. 36. Haralambos Sarimveis, Philip Doganis, Alex Alexandridis, “A classification technique based on radial basis function neural networks”, Advances in Engineering Software vol.37, pp.218–221, 2006. 37. Chauvin in, Y and D.E. Ruumehart Backpropagation : Theory, Architechtures and Applications, Erbaum Mahwah, NJ, ISBN : 080581258. PP 561, 1995. 38. Shaikh Abdul Hannan, V. D. Bhagile, R.R. Manza, R. J. Ramteke, “Heart Disease Diagnosis By Using FFBP algorithm of Artificial Neural Network”, International Conference on Communication, Computation, Control and Nanotechnology, ICN-2010 Organized by Rural Engineering College Bhalki-585328, during October 29-30, 2010. 39. Cigizoglu HK, Alp M. Generalized regression neural network in modelling river sediment yield. Adv Eng Software 2005;37:63–8. 40. Kim B, Lee DW, Parka KY, Choi SR, Choi S. Prediction of plasma etching using a randomized generalized regression neural network. Vacuum 2004;76:37–43 41. Jang JSR, Sun CT, Mizutani E. Neuro-fuzzy and soft computing: a computational approach to learning and machine intelligence, Prentice Hall, Upper Saddle River, New Jersey, USA; 1997 Chapter 9. 42. Shaikh Abdul Hannan, R. R. Manza and R.J. Ramteke, “Association Rules for Filtering The Medicine To Avoid Side Effects Of Heart Patients”, on 16 -19 Dec 2009, at Advances in Computer Vision and Information Technology – 09, Dr. Babasaheb Ambedkar Marathwada University, Aurangabad. 43. Anupriya Kamble, Abdul Hannan, Yogesh, Dnyaneshwari, “Association detection of Regular Insulin and NPH Insulin Using Statistical Features”, Second International Conference on Cognitive Knowledge Engineering, 21-23 December 2016 (ICKE-2016), Aurangabad, Maharashtra, India, pp 59-62, ISBN 978-93-80876-89-4. 44. Shaikh Abdul Hannan, Pravin Yannawar, R. R. Manza and R.J. Ramteke, “Data Mining Technique for Detection of Cardiac Problems Using Symptoms Medicine and Its Side effects”, in IT & Business Intelligence -09 , in IT & Business Intelligence, on 06-08 Nov 2009, Organized By IMT, Nagpur, India. 45. Shaikh Abdul Hannan, Jameel Ahmed, Naveed Ahmed, Rizwan Alam Thakur, “Data Mining and Natural Language Processing Methods for Extracting Opinions from Customer Reviews”, International Journal of Computational Intelligence and Information Security, pp 52-58, Vol. 3, No. 6, July 2012. ( ISSN: 1837-7823). 46. Ordonez C,” Association rule discovery with the train and test approach for heart disease prediction”, IEEE Transactions on Information Technology in Biomedicine, P(334 – 343), April 2006 47. Aqueel Ahmed, Shaikh Abdul Hannan, “Data Mining Techniques to Find Out Heart Diseases: An Overview”, International Journal of Innovative Technology and Exploring Engineering (IJITEE), An ISO 9001:2008 Certified International Journal, Volume-1, Issue-4, September 2012, ISSN: 2278-3075, New Delhi, India. 48. AbuKhousa, E “Predictive data mining to support clinical decisions: An overview of heart disease prediction systems”, IEEE Transaction on Innovations in Information Technology (IIT), pp(267 - 272) March 2012..
J. Lin, R. Nogueira, and A. Yates. (2020)cite arxiv:2010.06467Comment: Final preproduction version of volume in Synthesis Lectures on Human Language Technologies by Morgan & Claypool.
Q. Le, and T. Mikolov. Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, page 1188--1196. Bejing, China, PMLR, (June 2014)
M. Benjak, Y. Samayoa, and J. Ostermann. Proceedings of the 28th IEEE International Conference on Image Processing (ICIP), (September 2021)accepted for publication.
S. Wang, L. Hu, Y. Wang, X. He, Q. Sheng, M. Orgun, L. Cao, F. Ricci, and P. Yu. (2021)cite arxiv:2105.06339Comment: Accepted by IJCAI 2021 Survey Track, copyright is owned to IJCAI. The first systematic survey on graph learning based recommender systems. arXiv admin note: text overlap with arXiv:2004.11718.
M. Paris, and R. Jäschke. Proceedings of the 14th International Conference on Knowledge Science, Engineering and Management, volume 12816 of Lecture Notes in Artificial Intelligence, page 1--14. Springer, (2021)
M. Dacrema, P. Cremonesi, and D. Jannach. (2019)cite arxiv:1907.06902Comment: Source code available at: https://github.com/MaurizioFD/RecSys2019_DeepLearning_Evaluation.
P. Xia, S. Wu, and B. Van Durme. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), page 7516--7533. Association for Computational Linguistics, (November 2020)
R. Schwarzenberg, L. Raithel, and D. Harbecke. (2019)cite arxiv:1904.01500Comment: NAACL-HLT 2019 Workshop on Evaluating Vector Space Representations for NLP (RepEval).