Pickens was born Louis Burton Lindley, Jr. in Kingsburg, California, the son of Sally Mosher (née Turk) and Louis Bert Lindley, Sr. He was an excellent rider from age 4 and quit school to join the rodeo at 12. He was told that working in the rodeo would be "slim pickings" (very little money), giving him his name, but he did well and eventually became a well-known rodeo clown — one of the most dangerous jobs in live entertainment.

After 20 years on the rodeo circuit, his distinctive Oklahoma-Texas drawl (even though he was a lifelong Californian), his wide eyes and moon face and strong physical presence gained him a role in the western Rocky Mountain (1950) starring Errol Flynn. He appeared in many more westerns, playing both villains and comic sidekicks to the likes of Rex Allen.

Pickens appeared in dozens of films, including Old Oklahoma Plains (1952), Down Laredo Way (1953), One-Eyed Jacks (1961) with Marlon Brando, Dr. Strangelove (1964), Major Dundee (1965) with Charlton Heston, the remake of Stagecoach (1966; Pickens played the driver, portrayed in the 1939 film by Andy Devine), Never a Dull Moment (1968), The Cowboys (1972) with John Wayne, Ginger in the Morning (1974) with Fred Ward, Blazing Saddles (1974), Poor Pretty Eddy (1975), Rancho Deluxe (1975), The Getaway with Steve McQueen, Tom Horn (1980), also with McQueen, An Eye for an Eye (1966) and Pat Garrett and Billy the Kid (1973) in a small but memorable role. He also had a small role in Steven Spielberg's 1941 (1979) in scenes with Toshirō Mifune and Christopher Lee; during one scene, he names the objects that he has with himself, and sounds like he does in Dr. Strangelove when identifying his possessions. In 1978, Pickens lent his voice to theme park Silver Dollar City as a character named Rube Dugan, for a ride called "Rube Dugan's Diving Bell", The diving bell was a simulation ride that took passengers on a journey to the bottom of Lake Silver and back. The ride was in operation from 1978 to 1984. He also played werewolf sheriff Sam Newfield in The Howling (1981).

In 1960 he appeared in NBC's Overland Trail in the episode "Sour Annie" with fellow guest stars Mercedes McCambridge and Andrew Prine. Pickens appeared five times on NBC's Outlaws (1960-1962) western series as the character "Slim." The program, starring Barton MacLane, was the story of a U.S. marshal in Oklahoma Territory -- deputies played by Don Collier, Jock Gaynor and Bruce Yarnell -- and the outlaws that they pursued. In 1967 Pickens had a recurring role as the scout California Joe Milner on the ABC military western Custer, starring Wayne Maunder in the title role.

Pickens was offered the part of Dick Hallorann in Stanley Kubrick's 1980 adaptation of Stephen King's The Shining. He refused, saying that filming with Kubrick on Dr. Strangelove was too strenuous. He later relented, saying that he would appear in the film as long as Kubrick was contractually required to shoot Pickens' scenes in fewer than 100 takes a shot. However, the role went to Scatman Crothers.

The next year Pickens was in another western, playing the evil, limping bank robber in Walt Disney's The Apple Dumpling Gang. He provided the voice of B.O.B. in the late '70's Disney Sci-Fi thriller The Black Hole.

Pickens' most famous role was as B-52 pilot Major T.J. "King" Kong in Dr. Strangelove. Stanley Kubrick cast Pickens after his first choice for the role of Major Kong, Peter Sellers, sprained his ankle and was unable to perform. Pickens was chosen because his accent and comic sense were perfect for the role of Kong, a cartoonishly patriotic and gung-ho B-52 commander. Pickens was not told that the movie was a black comedy, but was instructed to play the role straight.[citation needed] He was also not given the script to the entire film, but only those portions in which he played a part. Pickens is best remembered for three scenes:[citation needed]

Pickens credited Dr. Strangelove as a turning point in his career. Previously he was "Hey you" on sets and afterward he was addressed as "Mr. Pickens." He has been quoted as commenting, "After Dr. Strangelove the roles, the dressing rooms, and the checks all started gettin' bigger." Pickens said he was amazed at the difference a single movie could make.