arXiv:2604.01221  (cs)
 



 

  Authors:  Zhe Yang ,  Shulin Tian ,  Kairui Hu ,  Shuai Liu ,  Hoang-Nhat Nguyen ,  Yichi Zhang ,  Zujin Guo ,  Mengying Yu ,  Zinan Zhang ,  Jingkang Yang ,  Chen Change Loy ,  Ziwei Liu               View a PDF of the paper titled HippoCamp: Benchmarking Contextual Agents on Personal Computers, by Zhe Yang and 11 other authors 
 View PDF 

 
         Abstract: We present HippoCamp, a new benchmark designed to evaluate agents' capabilities on multimodal file management. Unlike existing agent benchmarks that focus on tasks like web interaction, tool use, or software automation in generic settings, HippoCamp evaluates agents in user-centric environments to model individual user profiles and search massive personal files for context-aware reasoning. Our benchmark instantiates device-scale file systems over real-world profiles spanning diverse modalities, comprising 42.4 GB of data across over 2K real-world files. Building upon the raw files, we construct 581 QA pairs to assess agents' capabilities in search, evidence perception, and multi-step reasoning. To facilitate fine-grained analysis, we provide 46.1K densely annotated structured trajectories for step-wise failure diagnosis. We evaluate a wide range of state-of-the-art multimodal large language models (MLLMs) and agentic methods on HippoCamp. Our comprehensive experiments reveal a significant performance gap: even the most advanced commercial models achieve only 48.3% accuracy in user profiling, struggling particularly with long-horizon retrieval and cross-modal reasoning within dense personal file systems. Furthermore, our step-wise failure diagnosis identifies multimodal perception and evidence grounding as the primary bottlenecks. Ultimately, HippoCamp exposes the critical limitations of current agents in realistic, user-centric environments and provides a robust foundation for developing next-generation personal AI assistants.
 

 
 
            
       Comments: 
       Project Page:  this https URL  
     

       Subjects: 
       
         Artificial Intelligence (cs.AI) ; Computer Vision and Pattern Recognition (cs.CV) 
      
       Cite as: 
         arXiv:2604.01221  [cs.AI]  
     
     
         
       (or  
           arXiv:2604.01221v1  [cs.AI]  for this version)
       
     
     
         
                      https://doi.org/10.48550/arXiv.2604.01221  
           
             Focus to learn more 
           
           
           
                                arXiv-issued DOI via DataCite (pending registration) 
         
       
      
 


 
  

   Full-text links: 
  

    view license  
 
      
Current browse context:  cs.AI 



     
  

 new 
  |  
 recent 
  |   2026-04 

Change to browse by:
 
     cs  
     cs.CV  
 


 
   
    

        NASA ADS    Google Scholar  
        Semantic Scholar  
     
      
   


 export BibTeX citation 
 Loading... 



 
     
        

         × 
     
     
         loading... 
     
     
         Data provided by:  
          
     
 

 Bibliographic Tools 
 
  

           Bibliographic Explorer Toggle 
         
       
       
         Bibliographic Explorer   ( What is the Explorer? ) 
       
     
     
       
         
           
            
           Connected Papers Toggle 
         
       
       
         Connected Papers   ( What is Connected Papers? ) 
       
      
       
         
           
            
           Litmaps Toggle 
         
       
       
         Litmaps   ( What is Litmaps? ) 
       
     
     
       
         
           
            
           scite.ai Toggle 
         
       
       
         scite Smart Citations   ( What are Smart Citations? ) 
       
     
   
      
      
      
      
 

 
 Code, Data, Media 
 
  

           alphaXiv Toggle 
         
       
       
         alphaXiv   ( What is alphaXiv? ) 
       
     

     
       
         
           
            
           Links to Code Toggle 
         
       
       
         CatalyzeX Code Finder for Papers   ( What is CatalyzeX? ) 
       
     

     
       
         
           
            
           DagsHub Toggle 
         
       
       
         DagsHub   ( What is DagsHub? ) 
       
     

     
       
         
           
            
           GotitPub Toggle 
         
       
       
         Gotit.pub   ( What is GotitPub? ) 
       
     

     
       
         
           
            
           Huggingface Toggle 
         
       
       
         Hugging Face   ( What is Huggingface? ) 
       
     

     
       
         
           
            
           Links to Code Toggle 
         
       
       
         Papers with Code   ( What is Papers with Code? ) 
       
     

     
       
         
           
            
           ScienceCast Toggle 
         
       
       
         ScienceCast   ( What is ScienceCast? ) 
       
     
   

    
    
    
    
    
    
    
    
 

   
   Demos 
   
    

             Replicate Toggle 
           
         
         
           Replicate   ( What is Replicate? ) 
         
       
       
         
           
             
              
             Spaces Toggle 
           
         
         
           Hugging Face Spaces   ( What is Spaces? ) 
         
       
       
         
           
             
              
             Spaces Toggle 
           
         
         
           TXYZ.AI   ( What is TXYZ.AI? ) 
         
       
     
      
      
      
   
   
   Related Papers 
   
    

             Link to Influence Flower 
           
         
         
           Influence Flower   ( What are Influence Flowers? ) 
         
       
       
         
           
             
              
             Core recommender toggle 
           
         
         
           CORE Recommender   ( What is CORE? ) 
         
        
      
     
       
          Author  
          Venue  
          Institution  
          Topic  
       
       
            
            
            
            
       
     
      
      
   

   
   
    About arXivLabs
   
   
     
       
        

         arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

        Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

        Have an idea for a project that will add value for arXiv's community?   Learn more about arXivLabs  .

       
       
        

       
     
   

 



 Which authors of this paper are endorsers?  |
 Disable MathJax  ( What is MathJax? )