Abstract: A modeling method was proposed based on a novel image-based framework to profile and model the cell dynamics in live-cell videos. In the framework, the cell dynamics between frames were represented as frame-level features about cell deformation and intracellular movement. The cell deformation was captured by the shape context, while the intracellular movement was modeled with SIFT (Scale-Invariant Feature Transform) flow. In order to completely evaluate the streaming of protoplasm,an appearance change field was constructed on the basis of the displacement field. Then time series modeling was performed for these frame-level cell dynamic features. Specifically, temporal feature aggregation, and compact encoding in particular, was applied to capturing the video-wide temporal evolution of cell dynamics. A cell-live video dataset was developed to validate the effectiveness of the proposed framework. The experimental results demonstrate that, the proposed method is better than other mainstreaming approaches in measuring and clustering the cell deformation dynamics.