{
    "abstract": "Retrieving keyframes most relevant to text from small intestine videos with given labels can efficiently and accurately locate pathological regions. However, training directly on raw video data is extremely slow, while learning visual representations from image-text datasets leads to computational inconsistency. To tackle this challenge, a small bowel video keyframe retrieval based on multi-modal contrastive learning (KRCL) is proposed. This framework fully utilizes textual information from video category labels to learn video features closely related to text, while modeling temporal information within a pretrained image-text model. It transfers knowledge learned from image-text multimodal models to the video domain, enabling interaction among medical videos, images, and text data. Experimental results on the hyper-spectral and Kvasir dataset for gastrointestinal disease detection (Hyper-Kvasir) and the Microsoft Research video-to-text (MSR-VTT) retrieval dataset demonstrate the effectiveness and robustness of KRCL, with the proposed method achieving state-of-the-art performance across nearly all evaluation metrics.\n\u4e3a\u7ed9\u5b9a\u6807\u7b7e\u7684\u5c0f\u80a0\u89c6\u9891\u68c0\u7d22\u51fa\u4e0e\u6587\u672c\u6700\u76f8\u5173\u7684\u5173\u952e\u5e27\u753b\u9762\uff0c\u53ef\u4ee5\u9ad8\u6548\u7cbe\u51c6\u5730\u5b9a\u4f4d\u75c5\u53d8\u4f4d\u7f6e\u3002\u7136\u800c\uff0c\u5982\u679c\u76f4\u63a5\u4ece\u539f\u89c6\u9891\u4e2d\u5b66\u4e60\uff0c\u8bad\u7ec3\u8fc7\u7a0b\u4f1a\u975e\u5e38\u7f13\u6162\uff1b\u800c\u5982\u679c\u4ece\u56fe\u50cf\u2014\u6587\u672c\u6570\u636e\u96c6\u4e2d\u5b66\u4e60\u89c6\u89c9\u8868\u5f81\uff0c\u5219\u4f1a\u5bfc\u81f4\u8ba1\u7b97\u7684\u4e0d\u4e00\u81f4\u6027\u3002\u4e3a\u4e86\u5e94\u5bf9\u8fd9\u4e00\u6311\u6218\uff0c\u672c\u6587\u57fa\u4e8e\u591a\u6a21\u6001\u5bf9\u6bd4\u5b66\u4e60\uff0c\u63d0\u51fa\u4e86\u5c0f\u80a0\u89c6\u9891\u5173\u952e\u5e27\u68c0\u7d22\uff08KRCL\uff09\u6846\u67b6\u3002\u8be5\u6846\u67b6\u5145\u5206\u5229\u7528\u89c6\u9891\u7c7b\u522b\u6807\u7b7e\u4e2d\u7684\u6587\u672c\u4fe1\u606f\uff0c\u4ee5\u5b66\u4e60\u4e0e\u6587\u672c\u5bc6\u5207\u76f8\u5173\u7684\u89c6\u9891\u7279\u5f81\uff1b\u540c\u65f6\u5b9e\u73b0\u5728\u9884\u8bad\u7ec3\u7684\u56fe\u50cf\u2014\u6587\u672c\u6a21\u578b\u4e2d\u5efa\u6a21\u89c6\u9891\u7684\u65f6\u5e8f\u4fe1\u606f\uff0c\u5c06\u56fe\u50cf\u2014\u6587\u672c\u591a\u6a21\u6001\u6a21\u578b\u4e2d\u5b66\u5230\u7684\u77e5\u8bc6\u8fc1\u79fb\u5230\u89c6\u9891\u9886\u57df\uff0c\u5b9e\u73b0\u533b\u5b66\u89c6\u9891\u3001\u56fe\u50cf\u548c\u6587\u672c\u6570\u636e\u4e4b\u95f4\u7684\u4ea4\u4e92\u3002\u5728\u9ad8\u7ea7\u865a\u62df\u5185\u7aa5\u955c\u514b\u74e6\u5e0c\u5c14\u80c3\u90e8\u75c5\u53d8\u8bca\u65ad\uff08Hyper-Kvasir\uff09\u6570\u636e\u96c6\u548c\u5fae\u8f6f\u7814\u7a76\u9662\u89c6\u9891\u2014\u6587\u672c\u68c0\u7d22\uff08MSR-VTT\uff09\u6570\u636e\u96c6\u4e0a\u7684\u5b9e\u9a8c\u7ed3\u679c\u5206\u522b\u8bc1\u660e\u4e86KRCL\u7684\u6709\u6548\u6027\u548c\u9c81\u68d2\u6027\uff0c\u5e76\u4e14\u672c\u6587\u6240\u63d0\u51fa\u7684\u65b9\u6cd5\u5728\u51e0\u4e4e\u6240\u6709\u7684\u8bc4\u4ef7\u6307\u6807\u4e0a\u8fbe\u5230\u4e86\u76ee\u524d\u6700\u5148\u8fdb\u7684\u6027\u80fd\u3002.",
    "authors": [
        {
            "affiliation": "School of Computer Engineering and Science, Shanghai University, Shanghai 200444, P. R. China.\nShanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai 200444, P. R. China.",
            "firstname": "Xing",
            "initials": "X",
            "lastname": "Wu"
        },
        {
            "affiliation": "School of Computer Engineering and Science, Shanghai University, Shanghai 200444, P. R. China.",
            "firstname": "Guoyin",
            "initials": "G",
            "lastname": "Yang"
        },
        {
            "affiliation": "School of Computer Engineering and Science, Shanghai University, Shanghai 200444, P. R. China.",
            "firstname": "Jingwen",
            "initials": "J",
            "lastname": "Li"
        },
        {
            "affiliation": "Medical College of Shanghai University, Shanghai University, Shanghai 200444, P. R. China.\nShanghai Universal Medical Imaging Diagnostic Center, Shanghai 200233, P. R. China.",
            "firstname": "Jian",
            "initials": "J",
            "lastname": "Zhang"
        },
        {
            "affiliation": "Gastroenterology, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiaotong University, Shanghai 200233, P. R. China.",
            "firstname": "Qun",
            "initials": "Q",
            "lastname": "Sun"
        },
        {
            "affiliation": "Faculty of Science, Yamaguchi University, Yamaguchi-Ken 753-8511, Japan.",
            "firstname": "Xianhua",
            "initials": "X",
            "lastname": "Han"
        },
        {
            "affiliation": "School of Computer Engineering and Science, Shanghai University, Shanghai 200444, P. R. China.\nShanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai 200444, P. R. China.",
            "firstname": "Quan",
            "initials": "Q",
            "lastname": "Qian"
        },
        {
            "affiliation": "College of Information Science and Engineering, Ritsumeikan University, Shiga-Ken 525-8577, Japan.",
            "firstname": "Yanwei",
            "initials": "Y",
            "lastname": "Chen"
        }
    ],
    "conclusions": null,
    "copyrights": null,
    "doi": "10.7507/1001-5515.202304021",
    "journal": "Sheng wu yi xue gong cheng xue za zhi = Journal of biomedical engineering = Shengwu yixue gongchengxue zazhi",
    "keywords": [
        "Contrastive learning",
        "Multi-modal learning",
        "Video keyframe retrieval"
    ],
    "methods": null,
    "publication_date": "2025-04-28",
    "pubmed_id": "40288976",
    "results": null,
    "title": "[Small bowel video keyframe retrieval based on multi-modal contrastive learning].",
    "xml": "<Element 'PubmedArticle' at 0x788505f3e200>"
}