With the expansion of the virtual communication industry using VR/AR, it has attracted increasing attention to enable users to represent their personalities in a 3D avatar. As the face of 3D avatars plays a crucial role in conveying human personality, a system that generates and manipulates 3D faces is desired. However, establishing the system is challenging due to the need for human effort and specialized knowledge. To fill this void, we present the Individual Virtual Transfer (InViTe), which enables the creation and customization of a 3D face according to the user's preference. Our proposed system is featured for 1) 3D face reconstruction with high fidelity texture map, 2) 3D face personalization, 3) realistic rendering results, and 4) real-time mobile virtual applications. We conduct an experiment to demonstrate that the proposed system can achieve sufficient individual personalization of 3D faces. Furthermore, we evaluate the system's data transmission protocol and demonstrate its efficiency. The demonstration video is available at https://www.youtube.com/watch?v=D 4pXZvGUWU.
论文检索
输入标题、作者或关键词，从 1,048 篇学术成果中精准定位
Influence diffusion models are used for simulating information propagation in social networks. While most existing influence diffusion models are probabilistic, the emergence of Large Language Model (LLM) sheds light on the language-level inferences and interactions of user agents. This paper presents an LLM-enhanced Agent-based Influence Diffusion model (LAID), and a web-based visualization tool, LAIDSim, for simulating the information propagation in social networks.
We introduce an open-source platform for annotating body-worn video (BWV) footage aimed at enhancing transparency and accountability in policing. Despite the widespread adoption of BWVs in police departments, analyzing the vast amount of footage generated has presented significant challenges. This is primarily due to resource constraints, the sensitive nature of the data, which limits widespread access, and consequently, lack of annotations for training machine learning models. Our platform, called CVAT-BWV, offers a secure, locally hosted annotation environment that integrates several AI tools to assist in annotating multimodal data. With features such as automatic speech recognition, speaker diarization, object detection, and face recognition, CVAT-BWV aims to reduce the manual annotation workload, improve annotation quality, and allow for capturing perspectives from a diverse population of annotators. This tool aims to streamline the collection of annotations and the building of models, enhancing the use of BWV data for oversight and learning purposes to uncover insights into police-civilian interactions.
Today industrial facilities are equipped with lots of sensors throughout all the production line for monitoring means. Gathered data can be used to detect and predict failures; however, manual labeling of large amounts of data for supervised learning is complicated. This paper introduces an innovative approach to unsupervised fault detection and diagnosis tailored for monitoring industrial chemical processes. We showcase the efficacy of our model using two publicly accessible datasets from the Tennessee Eastman Process, each containing various faults. Furthermore, we illustrate that by fine-tuning the model on a limited amount of labeled data, it achieves performance close to that of a state-of-the-art model trained on the entire dataset.
Manufacturing companies are experiencing a transformative journey, moving from labor-intensive processes to integrating cutting-edge technologies such as digitalization and AI. In this demo paper, we present a novel AI tool to enhance manufacturing processes. Remarkably, our work has been developed in collaboration with Agrati S.p.A., a worldwide leading company in the bolts manufacturing sector. In particular, we propose an AI-powered tool to address the problem of automatically generating the production cycle of a bolt. Currently, this decision-making task is performed by process engineers who spend several days to study, draw, and test multiple alternatives before finding the desired production cycle. We cast this task as a model-based planning problem, mapping bolt technical drawings and metal deformations to, potentially continuous, states and actions, respectively. Furthermore, we resort to computer vision tools and visual transformers to design efficient heuristics that make the search affordable in concrete applications. Agrati S.p.A.'s process engineers extensively validated our tool, and they are currently using it to support their work. To the best of our knowledge, ours is the first AI tool dealing with production cycle design in bolt manufacturing.
Research in 3D mapping is crucial for smart city applications, yet the cost of acquiring 3D data often hinders progress. Visual localization, particularly monocular camera position estimation, offers a solution by determining the camera's pose solely through visual cues. However, this task is challenging due to limited data from a single camera. To tackle these challenges, we organized the AISG–SLA Visual Localization Challenge (VLC) at IJCAI 2023 to explore how AI can accurately extract camera pose data from 2D images in 3D space. The challenge attracted over 300 participants worldwide, forming 50+ teams. Winning teams achieved high accuracy in pose estimation using images from a car-mounted camera with low frame rates. The VLC dataset is available for research purposes upon request via vlc-dataset@aisingapore.org.
GraphQL offers an efficient, powerful, and flexible alternative to REST APIs. However, application developers writing GraphQL clients need both technical and domain-specific expertise to reap its benefits, and avoid over-fetching or under-fetching data. Automated GraphQL generation has so far proven to be a hard problem because of complex GraphQL schema and lack of benchmark datasets. To address these issues, our work focuses on building an LLM-powered pipeline that can accept user requirements in natural language along with the complex GraphQL schema and automatically produce the GraphQL query needed to retrieve the necessary data. Automated GraphQL generation helps reduce entry barriers to application developers, broadening GraphQL adoption.
This paper demonstrates a novel method for LiDAR-based 3D object detection, addressing major field challenges: sparsity and occlusion. Our approach leverages temporal point cloud sequences to generate frames that provide comprehensive views of objects from multiple angles. To address the challenge of generating these frames in real-time, we employ Knowledge Distillation within a Teacher-Student framework, allowing the Student model to emulate the Teacher’s advanced perception. We pioneered the application of weak-to-strong generalization in computer vision by training our Teacher model on enriched, object-complete data. In this demo, we showcase the exceptional quality of labels produced by the X-Ray Teacher on object-complete frames, showing our method distilling its knowledge to enhance object 3D detection models.
In this work, we propose a novel framework for image segmentation guided by visual prompting which leverages the power of vision foundation models. Inspired by recent advancements in computer vision, our approach integrates multiple large-scale pretrained models to address the challenges of segmentation tasks with limited and sparsely annotated data interactively provided by a user. Our method combines a frozen feature extraction backbone with a scalable and efficient probabilistic feature correspondence (soft matching) procedure derived from Optimal Transport to couple pixels between reference and target images. Moreover, a pretrained segmentation model is harnessed to translate user scribbles into reference masks and matched target pixels into output target segmentation masks. This results in a framework that we name Softmatcher, a versatile and fast training-free architecture for image segmentation by visual prompting. We demonstrate the efficiency and scalability of Softmatcher for real-time interactive image segmentation by visual prompting and showcase it in diverse visual domains including technical visual inspection use cases.
An Interactive Human-Machine Learning Interface for Collecting and Learning from Complex Annotations
PDF ↗Human-Computer Interaction has been shown to lead to improvements in machine learning systems by boosting model performance, accelerating learning and building user confidence. In this work, we aim to alleviate the expectation that human annotators adapt to the constraints imposed by traditional labels by allowing for extra flexibility in the form that supervision information is collected. For this, we propose a human-machine learning interface for binary classification tasks which enables human annotators to utilise counterfactual examples to complement standard binary labels as annotations for a dataset. Finally we discuss the challenges in future extensions of this work.
NegoLog: An Integrated Python-based Automated Negotiation Framework with Enhanced Assessment Components
PDF ↗The complexity of automated negotiation research calls for dedicated, user-friendly research frameworks that facilitate advanced analytics, comprehensive loggers, visualization tools, and auto-generated domains and preference profiles. This paper introduces NegoLog, a platform that provides advanced and customizable analysis modules to agent developers for exhaustive performance evaluation. NegoLog introduces an automated scenario and tournament generation tool in its Web-based user interface so that the agent developers can adjust the competitiveness and complexity of the negotiations. One of the key novelties of the NegoLog is an individual assessment of preference estimation models independent of the strategies.
Molecular graph representation learning plays a crucial role in various domains, such as drug discovery and chemical reaction prediction, where molecular graphs are typically depicted as 2D topological structures. However, recent insights highlight the critical role of 3D geometric information and functional groups in accurately predicting molecular properties, aspects often neglected in existing molecular graph benchmark datasets. To bridge the research gap, we introduce a comprehensive molecular learning benchmark named 3D-FUM, which incorporates both 3D geometric information and functional groups of a large number of molecules. 3D-FUM integrates 18 state-of-the-art algorithms and 19 evaluation metrics on three molecular learning tasks, including general molecule generation, conditional molecule generation, and property predictions. 3D-FUM, for the first time, take into consideration both 3D geometric information and molecular functional groups, which enables researchers and practitioners to effectively and impartially evaluate newly proposed methods in comparison to existing baselines across diverse datasets. Furthermore, we design a user interface for user-friendly interaction and development with the benchmark for evaluation metrics selection, parameter adjustment, and leaderboard comparison. To ensure accessibility and reproducibility, we opensource our benchmark 3D-FUM and experimental results at https://3dfunctiongroupmoleculedataset.github.io/3D-FuM/#/Home.
To optimize the preparation process for educators in academic lectures and associated question-and-answer sessions, this paper presents E-QGen, a lecture abstract-based question generation system. Given a lecture abstract, E-QGen generates potential student inquiries. The questions suggested by our system are expected to not only facilitate teachers in preparing answers in advance but also enable them to supply additional resources when necessary.
Integrating LLM, VLM, and Text-to-Image Models for Enhanced Information Graphics: A Methodology for Accurate and Visually Engaging Visualizations
PDF ↗This study presents an innovative approach to the creation of information graphics, where the accuracy of content and aesthetic appeal are of paramount importance. Traditional methods often struggle to balance these two aspects, particularly in complex visualizations like phylogenetic trees. Our methodology integrates the strengths of Large Language Models (LLMs), Vision Language Models (VLMs), and advanced text-to-image models to address this challenge. Initially, an LLM plans the layout and structure, employing Mermaid—a JavaScript-based tool that uses Markdown-like scripts for diagramming—to establish a precise and structured foundation. This structured script is crucial for ensuring data accuracy in the graphical representation. Following this, text-to-image models are employed to enhance the vector graphic generated by Mermaid, adding rich visual elements and enhancing overall aesthetic appeal. The integration of text-to-image models is a key innovation, enabling the creation of graphics that are not only informative but also visually captivating. Finally, a VLM performs quality control, ensuring that the visual enhancements align with the informational accuracy. This comprehensive approach effectively combines the accuracy of structured data representation, the creative potential of text-to-image models, and the validation capabilities of VLMs. The result is a new standard in information graphic creation, suitable for diverse applications ranging from education to scientific communication, where both information integrity and visual engagement are essential.
Reliable detection of human intentions from electroencephalogram (EEG) to improve human-robot interaction (HRI) has recently gained significant importance. To ensure safe and satisfactory interactions, implicit detection of erroneous behavior of robotic systems, particularly assistive devices, is essential. This can be achieved by detecting error-related potentials (ErrPs) in EEG, evoked by visual, tactile, or visuo-tactile stimuli. Of these, the ErrPs evoked tactilely with the help of a robot remains unexplored and has been the main focus of this competition. The task for participating teams was to develop robust AI models for continuous real-time classification of erroneous behavior of assistive robotic devices from the human EEG. Even though the competition results prove its feasibility, a performance gap (balanced accuracy and computation time) of more than 10% was observed between the offline and online classification of errors in real-world scenarios. In addition to the competitive AI models developed by the participating teams, this competition also contributed towards a one-of-its-kind open-access EEG and EMG dataset, a lossless live streaming solution for EEG data, and a novel quantitative metric for benchmarking online asynchronous EEG detection solutions.
FD-UAD: Unsupervised Anomaly Detection Platform Based on Defect Autonomous Imaging and Enhancement
PDF ↗In industrial quality control, detecting defects is essential. However, manual checks and machine vision encounter challenges in complex conditions, as defects vary among products made of different materials and shapes. We create FD-UAD, Unsupervised Anomaly Detection Platform Based on Defect Autonomous Imaging and Enhancement. It uses multi-sensor technology, combining RGB and infrared imaging, liquid lenses for adjustable focal lengths, and uses image fusion to capture multidimensional features. The system incorporates image restoration techniques such as enhancement, deblurring, denoising, and super-resolution, alongside unsupervised anomaly detection model for enhanced accuracy. FD-UAD is successfully used in a top diesel engine manufacturer, demonstrating its value in AI-enhanced industrial applications.
Building an intelligent and efficient medical assistant is still a challenging AI problem. The major limitation comes from the data modality scarceness, which reduces comprehensive patient perception. This demo paper presents GigaPevt, the first multimodal medical assistant that combines the dialog capabilities of large language models with specialized medical models. Such an approach shows immediate advantages in dialog quality and metric performance, with a 1.18% accuracy improvement in the question-answering task.
Rule mining finds patterns in structured data such as knowledge graphs. Rules can predict facts, help correct errors, and yield explainable insights about the data. However, existing rule mining implementations focus exclusively on mining rules -- and not on their application. The PyClause library offers a rich toolkit for the application of the mined rules: from explaining facts to predicting links, scoring rules, and deducing query results. The library is easy to use and can handle substantial data loads.
The localization problem of the United Nations Sustainable Development Goals (SDGs) involves adopting strategies that are in tune with local conditions, to achieve a given SDG target. However, even within a given region, localized conditions may vary drastically. With increasing amounts of Open Government Data (OGD) being available, there is an opportunity to systematically address the localization problem by using predictive and prescriptive modeling techniques. This work presents a predictive and prescriptive modeling dashboard for the SDG indicator maternal deaths (MD) for the Indian state of Karnataka. The dashboard was created by examining a vast set of data points to focus on four factors that showed high correlations with that of MD. We then construct a multivariate linear regression model to showcase the differential impact that a given factor has on the indicator and identify prescribed values for different factors to achieve a given target value of the indicator. Finally, a budget allocation dashboard is also provided that helps policymakers allocate budgets to specific schemes to help operationalize these changes. This dashboard was built by combining data coming from five different OGD sources.
PyXAI (Python eXplainable AI) is a Python library designed for providing explanations and cor- recting tree-based Machine Learning (ML) models. It is suited to decision trees, random forests, and boosted trees, when used for regression or classification tasks. In contrast to many model-agnostic approaches to XAI, PyXAI exploits the model it- self to generate explanations, ensuring them to be faithful. PyXAI includes several algorithms for the generation of explanations, which can be abductive or contrastive. PyXAI also includes algorithms for correcting tree-based models when their predictions conflict with pieces of user knowledge.