In this paper we propose a method for efficient extraction of human silhouette from video sequences. The proposed approach includes background elimination, edge detection, region filling and noise removal using morphological operations to estimate the silhouette of an image. To the best of our knowledge our proposed approach for silhouette extraction involving background elimination and edge detection is first of its kind. We have applied our proposed technique on Weizmann (standard) dataset and compared the results with the most recent related research work. The comparison results in terms of statistical measures like precision, recall and F-measure clearly show the supremacy of our method and thus justify its novelty. © Springer Nature Singapore Pte Ltd. 2017.